Despite the growing use of NLP in second language (L2) research, model accuracy in L2 settings remains underexplored. This study addresses this gap by evaluating and fine-tuning a Korean language model to extract morphosyntactic features (i.e., morpheme tokenization/tagging and dependency parsing) from L2-Korean texts. We begin by evaluating a domain‐general Korean language model on a gold‐annotated L2-Korean treebank. We then fine‐tune the model on L2-Korean data and quantify the resulting gains across diverse L1- and L2- datasets. Finally, we examine how model reliability varies with learner proficiency scores. Three key findings emerge: while the domain-general model excels at morpheme tokenization, it underperforms on morpheme tagging and dependency parsing; fine‐tuning substantially improves adaptability to L2 morphosyntax; and proficiency has minimal effect on morpheme‐level tasks but significantly affects dependency-parsing reliability. These results highlight the importance of incorporating L2 training data to improve morphosyntactic analysis in L2 settings and caution against uncritical reliance on automated dependency annotations, especially when performance varies across proficiency levels.
Sung et al. (Fri,) studied this question.