Towards Fully Autonomous Conversational Agents: A Comparative Evaluation of Whisper Models with Elderly Users.

Achieving fully autonomous verbal interaction in social robots for older adults requires Automatic Speech Recognition (ASR) systems capable of handling both the variability of real-world audio environments and the specific speech characteristics associated with aging. This study presents a comparative evaluation of five Whisper models (Tiny, Base, Small, Medium, and Large) applied to 39 audio samples collected during real-world sessions with older adult users participating in conversational cognitive stimulation games. Each model is tested with and without a prior noise filtering step using spectral gating, and its performance is evaluated using the Levenshtein distance against manually transcribed reference texts. In addition to transcription quality and inference time, a second evaluation layer is introduced: The transcribed texts are post-processed and interpreted using a large language model (GPT-4o), and the semantic fidelity of the generated response is compared to the original speaker’s intent. This two-level evaluation enables the analysis of not only ASR accuracy but also the robustness of the entire system in natural voice interaction flows between robots and older adults.