Automatic reflection level classification in Hungarian student essays

Reflective thinking is a key competency in education, but assessing reflective writing remains a time-consuming and subjective task for education experts. While automated reflective analysis has been explored in several languages, Hungarian language was not researched extensively. This study examines automatic reflection-level classification in Hungarian student essays. We used a large, expert-annotated Hungarian dataset consisting of 1954 reflective essays collected over multiple academic years and labeled on a four-level reflection scale. We investigate two approaches: (1) classical machine learning models using TF-IDF and semantic embedding features, and (2) Hungarian-specific transformer models fine-tuned for document-level reflection classification. Given class imbalance in the dataset, class weighting, oversampling, data augmentation, and alternative loss functions are evaluated systematically. An extensive ablation study is conducted to analyze the contribution of each modeling and balancing strategy. The results show that shallow machine learning models with appropriate feature engineering achieve strong overall performance, reaching up to 70.90% accuracy, 66.08% F1-score, and 74.57% ROC AUC score, while transformer-based models achieve slightly higher Accuracy and F1 (70.95% accuracy, 67.62% F1-score) but lower ROC AUC (67.60%), and demonstrate better generalization on minority reflection classes. These findings highlight the continued relevance of classical methods for low-resource settings and the robustness of transformer models for imbalanced classification. The experimental insights provide a solid foundation for future research on automated reflective analysis in Hungarian and other morphologically rich languages.