Data structures for storing language materials: principles and optimization

This article presents a universal approach to creating a specialized corpus of annotated data intended for training a model for information extraction from scientific literature in a specific realm. The process includes data collection and the development of annotation principles that take into account the linguistic features of academic texts and contextual characteristics. Both general programming data structures (arrays, lists, and trees) and specialized ones (corpora, lexicons, ontologies) adapted for solving linguistic issues are discussed. Particular attention is paid to optimization principles, including metadata unification, multi-level markup, ensuring representativeness, and supporting multimodality. The theoretical basis was provided by works on corpus linguistics, linguistic-semiotic foundations for studying scientific discourse, designing linguistic ontologies, and developing data structures for linguistic research. The methodology of automatic text processing is considered as an integral component of working with such structures, including the classification of methods (statistical, rule-based, machine learning) and analysis stages (morphological, syntactic, and semantic). The theoretical significance and practical value of the work lies in its contribution to the development of corpus linguistics, in terms of studying the possibilities and problems that arise in the process of corpus research. We note that the ability of data structures to cope with linguistic ambiguity and reflect complex relationships between linguistic elements using logical inference mechanisms and synergetic principles is an important criterion for the creation of intelligent systems. It is assumed that the trained model will be further used to automatically data extraction from a large unmarked literature array, which will form a knowledge graph of the subject area. Such a knowledge graph opens up opportunities for solving applied problems, including the compilation of frequency dictionaries for narrow scientific specialties and tracking trends in changes in terminology, including the emergence of new terminology.