Natural Language Processing (NLP) has emerged as a transformative tool for EFL speaking instruction. However, prior research lacks robust empirical investigations into how distinct NLP tools independently enhance adaptability, accuracy, and fluency—particularly through controlled, large-scale interventions. Most studies focus on short-term applications or conflate tool effects, leaving gaps in understanding mechanisms and sustained outcomes . This study examines how three separate NLP tools—AI chatbots, machine translation, and automatic summarization—independently influence EFL speaking skills, focusing on adaptability, accuracy, and fluency. A pretest-posttest randomized controlled trial (N = 436) assigned EFL learners to four groups: (1) Chatbot Treatment (20 role-play sessions via ChatGPT), (2) Machine Translation Treatment (bidirectional L1-L2 tasks using Google Translate/DeepL), (3) Automatic Summarization Treatment (SMMRY/QuillBot exercises), or (4) Control Group (traditional instruction) over 12 weeks. Chatbot Treatment produced the highest gains: adaptability (M = 85.50, Δ + 40.25), accuracy (M = 84.24, Δ + 43.93), and fluency (M = 85.04, Δ + 42.54; all p < .001). Pedagogically, educators should: (1) Integrate chatbots (e.g., ChatGPT, Replika) for structured conversational practice; (2) Use machine translation tools (e.g., DeepL) for vocabulary drills, not spontaneous speech; (3) Pair summarization tools (e.g., QuillBot) with explicit instruction on synthesizing ideas. Theoretically, findings demonstrate that chatbots uniquely optimize sociocultural learning mechanisms, enabling sustained fluency gains.
Zhang et al. (2025) studied this question.