Designing an LLM-driven mobile AR companion for social interaction support in children and youth with autism: a pilot study

Introduction Children and youth with autism spectrum disorder (ASD) often face persistent challenges in social interaction, communication, and participation in real-world contexts. Although individualized education programs and behavioral interventions play an important role in current practice, existing tools often lack the ability to provide continuous and adaptive support in everyday environments. To address this gap, this paper presents Stand By Me, an interactive system that integrates a large language model with mobile augmented reality. Methods The system embeds structured goals into everyday activities through an AR virtual pet, multimodal interaction, and personalized task generation, thereby supporting users’ interaction and task participation in real-world settings. Compared with traditional scripted training tools, the system dynamically generates responses based on user input and provides contextualized guidance by incorporating location and task information. This study explores the system through a descriptive real-world feasibility pilot conducted in collaboration with a rehabilitation institution. The pilot included 9 participants with ASD aged 6–22 years. Results The results showed that participants were able to repeatedly use the application during supervised training routines, interact with the AR virtual pet, and participate in real-world outing tasks with system guidance and caregiver supervision. Conclusion These findings provide preliminary feasibility evidence and design insights into the use of LLM-supported mobile AR systems for situated interaction and real-world task participation in ASD support contexts.