Exploring acoustic features and predictive modeling for automatic fluency assessment in Korean as a foreign language*

Research on fluency assessment in Korean as a foreign language (KFL) has primarily focused on assessment constructs and manual scoring, whereas studies on automatic fluency assessment remain limited. This study aims to explore effective fluency features for automatic fluency assessment of KFL learners and to compare the performance of different predictive models based on these features. Temporal (T5), acoustic (A8 and A88), and hybrid (H13 and H93) feature sets combining temporal and acoustic features were constructed and applied to tree-based models [Gradient Boosting Classifier (GBC) and XGBoost (XGB)] and transformer-based models for tabular data [FT-Transformer (FTT) and SAINT]. Experiments were conducted on 1,672,398 training and 209,083 validation utterances from the AIHub Korean Learner Speech Corpus. To examine the effect of data imbalance, performance was evaluated under original and balanced score-distribution conditions. Quadratic weighted kappa was the primary metric. Accuracy, Pearson’s correlation coefficient (r), root mean squared error, and Macro-F1 were supplementary metrics. Under original conditions, each model's acoustic condition outperformed its temporal and hybrid conditions. A8 was highest for GBC and XGB, A88 for FTT and SAINT, and SAINT-A88 overall. Hybrid sets showed no consistent improvements. Under balanced conditions, patterns varied, but SAINT-A88 remained the best-performing configuration. These findings indicate that performance varied across the feature conditions and predictive models examined.