Enhancing Conversation Smoothness in Language Learning Chatbots: An Evaluation of GPT4 for ASR Error Correction

The integration of natural language processing (NLP) technologies into educational applications has shown promising results, particularly in the language learning domain. Many spoken open-domain chatbots have been used as speaking partners, helping language learners improve their language skills. However, one of the significant challenges is the high word-error-rate (WER) when recognising non-native/non-fluent speech, which interrupts conversation flow and leads to disappointment for learners. This paper explores the use of GPT4 for ASR error correction in conversational settings. In addition to WER, we propose to use semantic textual similarity (STS) and next response sensibility (NRS) metrics to evaluate the impact of correction models on conversation smoothness. We find that transcriptions corrected by GPT4 lead to higher conversation smoothness, despite an increase in WER. GPT4 also outperforms standard error correction methods without the need for in-domain training data.