A meta-analysis of the effects of generative artificial intelligence on learning, cognitive, and linguistic outcomes

This meta-analysis examined the effect of generative artificial intelligence (GenAI) on students' cognitive, learning, and linguistic outcomes. A total of 68 effect sizes extracted from 58 independent primary studies were analyzed using a three-level random-effects model with cluster-robust variance estimation applied at the study level. Seven moderator variables were evaluated: outcome domain, educational level, GenAI tool type, experimental design, cultural orientation, course/subject domain, and comparison-condition type. The pooled effect was large, g = 0.864, 95% CI [0.720, 1.009], though heterogeneity was substantial (I² ≈ 88%) and the 95% prediction interval, [−0.213, 1.942], was wide enough to include a null or even negative effect in a new setting. Moderator tests, computed against the correct subgroup-weighted reference value and adjusted for multiple comparisons, identified outcome domain (Qb = 9.45, p = .009) and educational level (Qb = 6.50, p = .039) as significant at the conventional threshold, though neither survived Holm correction. Cultural orientation showed the largest raw contrast (vertical-collectivist g = 0.926 vs. horizontal-individualist g = 0.454; Qb = 4.11, p = .043) and remained significant in two of three confound-adjusted models ( p = .032 and p = .041) once subject domain, GenAI tool, experimental design, and small-study bias were held constant. Egger's regression indicated funnel-plot asymmetry ( p = .002), and PET-PEESE estimates suggested that, once corrected for small-study effects, the true effect could be considerably smaller (g ≈ 0.26–0.45) than the observed pooled estimate. Results support a positive average effect of GenAI on learning-related outcomes. However, the wide prediction interval and evidence of small-study inflation counsel caution in generalizing the point estimate, and the moderator pattern beyond outcome domain should be treated as provisional.