Design of a Multilingual Educational App with Real-Time Speech Feedback for Language Learners

This article reviews the design and assessment of a multilingual speech educational application which targets specific skills like pronunciation and speaking through real-time speech feedback. Language learning tools have to be more engaging and individualized owing to the explosion of digital learning resources that have stemmed from globalization. The app in question is designed using concepts of automatic speech recognition (ASR) and natural language processing (NLP) to evaluate users’ speech, provide corrective feedback, and refine content relevant to their proficiency level and progress. The system enables multiple languages and a variety of skill levels which users can select based on their preferences and needs, ensuring a broad user base. It describes the design process of a modular system architecture based on a multilingual ASR phoneme error detection that includes a learner model for individualized lessons. 120 English, Spanish, and Mandarin learners evaluated over the course of the study reported improved engagement with the app, but more importantly, pronounced improvements in pronunciation accuracy over traditional apps. The introduction suggests that the user's experience is enhanced by the implementation of real-time feedback systems on mobile language applications, thereby enabling more effective language learning. This paper addresses a gap in research in educational technology by exploring the application of processing strategies in the development of personalized and adaptive systems for language instruction regardless of cultural and language differences.