Vietnamese Legal News Classification: A Comparative Study of Transformer-based Models

This paper presents a comprehensive evaluation of transformer-based models for Vietnamese legal news article classification, introducing a large-scale benchmark dataset of 48,675 legal news articles from two authoritative Vietnamese legal journalism portals. The dataset spans five major categories: Social Affairs (Xa hoi), Current Events (Thoi su), Law (Phap luat), Public Security (An ninh trat tu), and Judiciary (Tu phap). Four fine-tuned transformer models—PhoBERT, XLM-RoBERTa, mBERT, and ViELECTRA—are systematically compared alongside Gemini 2.5 Flash evaluated under both zero-shot and few-shot prompting settings. Experimental results demonstrate that fine-tuned models, particularly PhoBERT and XLM-RoBERTa, achieve strong performance with 87% accuracy. The API-based LLM, while achieving approximately 60% accuracy on non-blocked samples, faced significant operational constraints due to content safety filters that generated a refusal rate of over 95% on legal crime-related content, making it unsuitable for high-volume production deployment. An extended error analysis including a confusion matrix and qualitative examples reveals that the Current Events (Thoi su) category is the primary source of misclassification. McNemar's test confirms statistically significant performance differences between most model pairs. These findings highlight the practical advantages of domain-specific fine-tuning for Vietnamese legal news processing and provide a reproducible benchmark for future research.