Domain-Specific Large Language Model Finetuning using a Model Assistant for Financial Text Summarization

The financial market and public opinion are cor-related. This means that changes in the financial market can result in changes to public opinion and changes to public opinion can result in changes to the financial market. Accordingly, it is essential for understanding and interacting with the financial market to gather text content from online sources and process it. As a result of the rapid growth of social media and other online sources, we have seen an exponential rise in data, particularly textual data, in recent years. It can be difficult for a person to read, let alone process, the massive volumes of data generated every day. This indicates that we need automated methods for processing textual data and extracting useful information. Automated text summarization is a method of shortening huge amounts of text without losing essential information. Transformers, which can efficiently manage and analyze textual data, are state-of-the-art text summarization models. However, developing such an automated text summarization model specialized in a domain (e.g. finance) can be challenging since we lack necessary domain-specific summarization datasets. In this work, we propose a pipeline for fully automating the finetuning of a text summarization model in a specific domain, namely cryptocurrency domain, without the involvement of human annotators. To this end, we introduce a novel method for self-improvement of text summarization models which relies on a model assistant which encodes domain knowledge, enabling finetuning text summarization models in specific domains in which we lack specific-domain summarization datasets. The proposed method is evaluated on a cryptocurrency-related text summarization problem and three well-known Large Language Models (LLMs) used for text summarization.