Enhancing Text Summarization in a Limited Data and Long Sample Setting

Abstract This thesis addresses the challenges of automatic text summarization in a limited amount of data and a significantly large sample size setting. The study introduces a novel dataset of Ukrainian legislative documents professionally translated into English. Comprising only 99 document-summary pairs with average lengths of over 20,000 and 1,300 tokens respectively, this dataset provides an ideal opportunity to evaluate and compare both extractive and abstractive summarization techniques under the data scarcity and document length constraints. The research findings reveal that abstractive summarization, even in a zero-shot setting, outperforms extractive methods, achieving higher scores across ROUGE and BERTScore metrics for legislative texts. Additionally, this study introduces a new text segmentation technique that automates the manual preprocessing of long, unstructured texts for abstractive summarization. Initial results demonstrating a 0.23 ROUGE score compared to human-annotated references, while may seem modest, indicate the feasibility of automating the generation of training data for fine-tuning summarization models at any scale, unlocking not only the ability to handle large-scale datasets but also to improve the efficiency of the models, encouraging further research into enhancing the algorithm.