AI-Powered Chatbots in Science Education: Meta-Analysis of Pedagogical Integration and Student Achievement

The integration of artificial intelligence (AI) in education has accelerated, yet its pedagogical impact remains uneven and theoretically underexplored, particularly in science education. Existing studies often emphasize technical features or user satisfaction, with limited focus on how instructional design and learning context shape learning outcomes. This meta-analysis evaluated the effectiveness of AI-powered chatbots in improving student achievement in science education and identified key moderating factors influencing their impact. Using PRISMA guidelines, 26 empirical studies published between 2020 and 2024 were systematically reviewed and analyzed with a random-effects model. The overall effect size was statistically significant and moderate (Hedges’ g=0.610, p< 0.001), suggesting that chatbot-supported instruction outperformed traditional methods in many cases. However, substantial heterogeneity was observed (I²=96.58%), indicating that effectiveness varied significantly based on socio-economic context, subject area, pedagogical design, and learner experience. Chatbots were most effective in lower-middle-income countries and in subjects like computer science and natural sciences, especially when implemented through scaffolded or personalized learning strategies. Gains in engagement and satisfaction were common, while effects on self-efficacy and navigation were mixed. These findings challenge uniform assumptions about AI’s role in education and call for theory-informed, context-sensitive integration strategies. Importantly, this study extends existing learning theories by showing that AI-driven dialogue systems act not merely as tools but as active mediators of both cognitive and affective processes. Future research should pursue longitudinal designs, hybrid human–AI teaching models, and ethical frameworks to guide equitable and sustainable implementation across educational contexts.