A meta-analysis examining AI-assisted L2 learning

Abstract Numerous quantitative studies have investigated how artificial intelligence (AI) impacts the development of second language (L2). While individual studies delve into the effects of AI interventions on L2 learning, a meta-analysis provides a comprehensive evaluation of AI’s effectiveness in second language acquisition (SLA). Despite the growing body of meta-analytical research in AI-assisted language learning, several potential moderators have not been thoroughly investigated in previous meta-analyses. This meta-analysis examines the effectiveness of AI-assisted L2 learning and analyzes factors that can influence the effectiveness. The analysis included 15 studies that involved a total of 2,156 participants and generated 53 effect sizes. After correcting for measurement and sampling error, AI-assisted L2 learning demonstrated a positive large effect with d = 1.167. The Q statistic suggested that the true effect sizes varied significantly across studies, which warranted conducting a theory-based moderator analysis. The results of the moderator analysis revealed that type of AI-assisted interactions was not a significant moderator affecting the effectiveness of AI-assisted L2 learning; AI-assisted L2 learning was more beneficial in developing receptive skills than productive skills; AI technologies excelled at building learners’ vocabulary skills compared to other language skills; the effectiveness of AI-assisted L2 learning was higher in an in-class context than in an out-of-class context; IMALL was more impactful for L2 learning than ICALL; and there was no significant difference in the effectiveness of AI technology intervention for L2 learning between K-12 and college learners.