Effects of AI-supported interventions on language performance: a multilevel meta-analysis of language-learning innovations

Purpose: This systematic review and meta-analysis synthesized controlled evidence on the effects of AI-supported interventions on direct language-performance outcomes in educational settings. Design/methodology/approach: Searches were conducted in Scopus and Web of Science. Eligible studies were required to include an AI-supported language-learning intervention, a comparison or control condition, and extractable quantitative information for calculating Hedges’ g. The primary synthesis focused on direct performance outcomes, including writing, speaking, reading, vocabulary, pronunciation, grammar, translation, and overall language achievement. Affective, motivational, and perception-based outcomes were excluded. The final synthesis included 43 effect sizes from 37 controlled intervention studies. Data were analyzed using a multilevel random-effects model. Findings: AI-supported interventions demonstrated a positive average effect on language performance. Sensitivity analyses indicated that this finding was robust to alternative dependency-handling strategies, baseline-equivalence restrictions, the exclusion of serious-risk studies, and the removal of the largest outlying effect. However, substantial heterogeneity remained, moderator analyses were exploratory, and small-study-effect diagnostics yielded mixed results. Originality/value: By distinguishing direct language-performance outcomes from affective and perception-based measures and accounting for dependencies among effect sizes, this study provides a focused synthesis of controlled evidence on AI-supported language-learning interventions. The findings indicate that these interventions can improve language performance on average, although their effectiveness should not be assumed to generalize uniformly across instructional contexts.