“It feels like hard work trying to talk to it”: Understanding How Older Adults with Mild Cognitive Impairment (MCI) Experience and Repair Interactional and Conversational Breakdowns with Voice Assistants (VAs)

Designing Conversational AI systems to support older adults with MCI necessitates robustness in handling complex and personalized interactions. In this study, we investigate how older adults with MCI experience, navigate and attempt to repair breakdowns while interacting with a voice-based AI system in their homes for medication management through the lens of interactional ”seams”. Through a 20-week in-home deployment with 7 participant dyads, we analyzed 844 interactions to identify interactional and conversational breakdowns and user repair strategies. Drawing on post-deployment interviews, we reflect on the nature of these breakdowns and older adults’ experiences of mitigating them. We further illustrate how older adults draw on their situated knowledge and environment to make sense of and recover from these disruptions, highlighting the cognitive effort required in doing so. Our findings emphasize the collaborative nature of interaction in human-AI contexts, and point to the need for voice assistants to better align with users’ expectations for memory, their routines and external resources in their environment. We conclude by discussing opportunities for voice assistants to integrate contextual knowledge from older adults’ sociotechnical environment and to facilitate more meaningful and user-centered interactions.