From manual classification to transformer-based language models: assessing the quality and consistency of historical convective event records

This article investigates whether Transformer-based language models could replace the labour-intensive, manual classification of convective weather phenomena found in written texts from the pre-instrumental measurement era. The training set is based on a corpus of 6999 written observations from 494 Central European sources, spanning the period from 1000 to 1817. This corpus has been linguistically normalised, ranging from Middle High German to Contemporary German. For the training set, the text sources containing observations of thunderstorm and hail events are first classified manually using a formalised procedure. Quality assurance is initially carried out at the level of the existing textual information. To this end, evidence classes are introduced. These are based on linguistic evidence regarding the associated phenomena of thunderstorm and hail events. In the next step, the plausibility of the classified thunderstorm and hail events is assessed by checking their consistency with the DWD normal periods (1961–1990, 1991–2020). The seasonal signal is preserved across four language stages and nine source types, from the early 15th to the early 19th century. The classified thunderstorm and hail events exhibit a plausible physical signal and show strong correlations with current observations. The trained models “ThunderstormBERT” and “HailBERT” achieve macro-F1 scores of 0.83 and 0.93 respectively. Misclassifications primarily impact the middle thunderstorm class (moderate thunderstorm), where the classification scheme is least clear-cut.