Fine-Tuning and Prompt Tuning of the Gemma2 Model for Efficient Korean Document Summarization

This paper proposes fine-tuning and prompt tuning methods to enhance the Korean document summarization performance of the Gemma2-9B-IT model. The Gemma2 model was primarily pre-trained on English data, resulting in limited performance on non-English languages such as Korean. To address this limitation, we preprocess the Korean 'Document Summarization Text' dataset provided by AI Hub and apply parameter-efficient fine-tuning using the Unsloth library and LoRA to enable memory-efficient training. Additionally, we design an optimized prompt: “Generate a concise and natural summary that includes only the essential information,” to further improve summarization quality. For performance evaluation, we utilize BERTScore and RDASS, which rely on semantic embeddings. Experimental results show that the proposed approach outperforms the base model by 1.68% in BERTScore and 34% in RDASS. An ablation study reveals that fine-tuning contributes 20.18% and prompt tuning 9.39% to the overall improvement, with some overlapping effects observed between the two techniques. Notably, the model demonstrates substantial improvement in semantic similarity between the generated summary and the reference summary, although the response time increases by 56%. This study presents an effective methodology for adapting large language models to specific languages and domains, and provides systematic insights into the individual and combined effects of fine-tuning and prompt tuning, offering valuable guidance for optimization strategies in various languages and tasks.