What question did this study set out to answer?

This research aims to evaluate the effectiveness of a large language model-driven spoken dialogue system compared to a human interlocutor in assessing second language speaking performance.

July 8, 2026Language Testing

Human or Machine? Evaluating Second Language Speaking Performance in Paired Discussions with a Large Language Model-Driven Spoken Dialogue System vs. a Human Interlocutor

View Full Paper

Ask AI

Bookmark

Authors

SMShangchao MinZhejiang International Studies University ZHZhuohan HouZhejiang University YWYanxin WangZhejiang University

Discussion

Member takes

Overview

Randomized trial evaluates speaking performance in dialogues with a language model versus a human, suggesting implications for assessment use.

Key Points

This research aims to evaluate the effectiveness of a large language model-driven spoken dialogue system compared to a human interlocutor in assessing second language speaking performance.
Within-participant design with 30 participants
Comparison of interactions with a self-built large language model-driven spoken dialogue system (E-Talk) and a human interlocutor
Assessment of test scores and linguistic features during paired discussions.
No significant differences in overall oral performance between LLM-driven SDS and human interlocutor
LLM-driven SDS interactions showed slightly lower scores for pronunciation and language use, reduced lexical diversity, and slower speech rate
Increased syntactic complexity, greater initiative in discussions, and weaker interactive listening noted in LLM-driven segments.

Cite This Study

Min et al. (2026) studied this question.

synapsesocial.com/papers/6a4de976d2ea289ef6283965 https://doi.org/10.1177/02655322261457566

View Full Paper

Ask AI

Bookmark

July 8, 2026Language Testing

Human or Machine? Evaluating Second Language Speaking Performance in Paired Discussions with a Large Language Model-Driven Spoken Dialogue System vs. a Human Interlocutor

View Full Paper

Ask AI

Bookmark

Authors

SMShangchao MinZhejiang International Studies University ZHZhuohan HouZhejiang University YWYanxin WangZhejiang University

Discussion

Member takes

Overview

Randomized trial evaluates speaking performance in dialogues with a language model versus a human, suggesting implications for assessment use.

Key Points

This research aims to evaluate the effectiveness of a large language model-driven spoken dialogue system compared to a human interlocutor in assessing second language speaking performance.
Within-participant design with 30 participants
Comparison of interactions with a self-built large language model-driven spoken dialogue system (E-Talk) and a human interlocutor
Assessment of test scores and linguistic features during paired discussions.
No significant differences in overall oral performance between LLM-driven SDS and human interlocutor
LLM-driven SDS interactions showed slightly lower scores for pronunciation and language use, reduced lexical diversity, and slower speech rate
Increased syntactic complexity, greater initiative in discussions, and weaker interactive listening noted in LLM-driven segments.

Cite This Study

Min et al. (2026) studied this question.

synapsesocial.com/papers/6a4de976d2ea289ef6283965 https://doi.org/10.1177/02655322261457566

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

View Full Paper

Ask AI

Bookmark