A System for Processing Large Volumes of Text Information Using Neural Networks

This paper presents the implementation of a system for summarizing large amounts of text information using neural networks. Relevance of this system is due to the need to obtain a large amount of knowledge in a very short time. The system works through the telegram bot interface. Telegram bot provides the ability to generate an annotation for input text and output the result as text file. Programming language used is Python. PostgreSQL database management system was chosen to store the data. To implement the system, data was collected, with the help of which the model was further trained. A parser was implemented to collect data. Multilingual mBART model was chosen as the initial model. To evaluate the model of summarization problem, the ROUGE metric was used. The result is a system that allows to obtain reliable information in compressed, accessible form, namely in the form of a text file, for further analysis.