What question did this study set out to answer?

To systematically evaluate whether state-of-the-art large language models can effectively identify, correct, and pedagogically explain learner errors in English language education.

August 20, 20260 citationsOpen Access

Clause Encounters of the Third Kind: Can LLMs Replace Language Teachers?

KŠKristina Šekrst AKAna Kovačić

Key Points

To systematically evaluate whether state-of-the-art large language models can effectively identify, correct, and pedagogically explain learner errors in English language education.
Tested multiple large language models on learner error identification, correction, and explanation tasks under varying model parameters and with retrieval-augmented generation.
Evaluated model outputs using automated computational metrics (GLEU, BERTScore) in combination with expert human assessments for instructional appropriateness, linguistic nuance, and cultural sensitivity.
Models demonstrated proficient surface-level error identification and grammatical correction capabilities.
Explanations generated by the models frequently lacked the domain-specific linguistic terminology and pedagogical depth necessary for effective language instruction.

Abstract

While various organizations now actively encourage LLM use in classrooms, we still lack rigorous, systematic evaluations of how well these models actually perform the fundamental tasks of language pedagogy. This paper examines whether state-of-the-art LLMs can deliver the kind of corrective feedback and methodological explanations that language learners need. The study tests multiple large language models on their ability to identify, correct, and explain common learner mistakes in English, by systematically varying model parameters to investigate how these technical adjustments affect output quality, pedagogical clarity, and consistency, along with using retrieval-augmented generation to query methodological data. The evaluation employs automated metrics (GLEU, BERTScore) but also human expert judgments to capture dimensions that purely computational measures miss: linguistic nuance, cultural sensitivity, and instructional appropriateness. While models demonstrate impressive surface-level correction abilities, their explanations often lack the terminological and domain knowledge that effective language teaching requires, suggesting that current enthusiasm for AI-assisted language learning may be outpacing our understanding of these systems' actual pedagogical competence.

Ask AI

Helpful

Bookmark

View Full Paper