Annotating – understood here as the process in which segments of a text are marked as belonging to a defined category – can be seen as a key technique in many disciplines , especially for working with text in the Humanities e.g. Unsworth 2000, the Computational Sciences (e.g. ; ), and the Digital Humanities . In the field of Digital Humanities, annotations of text are utilized, among other purposes, for the enrichment of a corpus or digital edition with (linguistic) information (e.g. ; ), for close and distant reading methods (e.g. ), or for machine learning techniques (e.g. ). Defining categories to shape data has been used in different text analysis contexts, including the study of toponyms (e.g. ) and biographical data (e.g. ). The paper at hand showcases the use of annotations within the Vienna Time Machine project (2020-2022, PI: Claudia Resch) which aims to connect different knowledge resources about historical Vienna via Named Entity Recognition (NER). More specifically, it discusses the challenges and potentials of annotating 18th century death lists found in the Wiennerisches Diarium or Wiener Zeitung, an early modern newspaper which was first published in 1703 and has already been (partly) digitized in form of the so-called DIGITARIUM : Here, users can access over 330 high-quality full text issues of the newspaper which contain a number of different text types, including articles, advertisements and more structured texts, such as arrival or death lists. The focus of this article lies on the semi-structured death lists, which do not only appear in almost every issue of the historical Wiener Zeitung, but are also relatively consistent in their structure and display a high semantic density: Each entry contains detailed information about a deceased person, such as their name, occupation, place of death, and age. Annotating these semi-structured list items opens up multiple possibilities: The resulting classified data can be used for efficient distant or scalable reading, quantitative analyses , and as a gold standard for both rule-based and machine learning NER approaches (e.g. ). To reach this goal and as a first step of the annotation process, the project team conducted a close reading of various death lists from multiple decades to identify recurrent linguistic patterns and, based hereon, to develop a first expandable set of categories. This bottom-up approach resulted in five preliminary categories, namely person, occupation, place, cause-of-death, and age, which were color-coded and, accompanied by annotated examples, documented in the form of annotation guidelines as intersubjectively applicable and concise as possible. These guidelines were then used by two researchers familiar with the historic material to annotate a randomly drawn and temporally distributed sample of 500 death list entries in the browser-based environment Prodigy (https://prodi.gy). Hereby, the emphasis was put especially on emerging challenging cases, i.e. items where annotators were in doubt about their choice of category, the exact positioning of annotations or the necessity to annotate certain text segments at all. Whenever annotators encountered such ambiguous items, these were collected, grouped and – as a third step in the annotation process – discussed with an interdisciplinary group of linguists, historians and prosopographers. Within this collective, a solution for each group of issues was agreed on and incorporated into the annotation guidelines. Also, existing categories were revised where necessary. The new, more stable category system was then again used for a new sequence of annotation and discussion of ambiguities, resulting in an iterative process where annotation and category development became intertwined. This approach, explained in the article in more detail, demonstrates that tagsets are never entirely final, but always depend on particular knowledge interests and data material and that even the annotation of inherently semi-structured lists requires continuous critical reflection and considerable historical and linguistic knowledge. At the same time, it can be exemplified by this work that it is precisely these challenging cases which carry a great potential for gaining knowledge and can be considered central to the development of a valid annotation system (cf. ).
Resch et al. (Mon,) studied this question.