Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights

The advancement of natural language processing (NLP) has expanded the application of AI-based text classification in the legal domain. However, accurately classifying legal documents remains a challenging task due to the complexity of legal texts and the subtle differences between legal categories. This study conducts a comprehensive evaluation of various legal text classification models, ranging from traditional machine learning techniques to state-of-the-art large language models (LLMs), based on ten legal categories of sexual offense precedents. The experimental results demonstrate that fine-tuning small-scale models (Small LMs) such as KLUE-BERT on legal data yields superior performance compared to large general-purpose models such as GPT-3.5 and GPT-4.0, as well as traditional machine learning models. In particular, KLUE-BERT achieved the highest accuracy of 99.3%, indicating that domain adaptation and fine-tuning play a more crucial role in legal document classification than model size alone. Furthermore, we employed explainable AI (XAI) techniques to analyze the model’s predictions and conduct an in-depth review of misclassification cases. XAI-based analysis allowed us to identify key linguistic features influencing model decisions and revealed limitations in the model’s ability to fully capture subtle textual cues. To further validate these findings, we leveraged KICS data, which closely resembles real-world legal case records, as a testbed to evaluate the model’s generalization capabilities. The results indicate that the model struggles to interpret implicit contextual cues within legal texts. These findings emphasize the necessity for both high performance and interpretability in legal AI models. By utilizing XAI, this study proposes methods to enhance transparency in legal text classification, contributing to ongoing discussions on the reliability and practical application of AI in legal contexts. Our research suggests that AI-assisted tools can effectively support legal professionals in tasks such as legal document classification, legal information retrieval, and case assessment.