Exploring the Use of Large Language Model-Driven Chatbots in Virtual Reality to Train Autistic Individuals in Job Communication Skills

Autistic individuals commonly encounter challenges in communicating with others which can lead to difficulties in obtaining and maintaining jobs. Thus, job training programs have emphasized training the communication skills of autistic individuals to improve their employability. Hence, we developed a virtual reality application that features avatars as chatbots powered by Large Language Models (LLMs), such as GPT-3.5 Turbo, and employs speech-based interactions with users. The use of LLM-driven chatbots allows job coaches to create training scenarios for trainees using text prompts. We conducted a preliminary study with three autistic trainees and two job coaches to gather early-stage feedback on the application's usability and user experience. In the study, the trainee participants were asked to interact with the application in two scenarios involving customer interactions. Our findings indicate that our application shows promise for training job communication. Furthermore, we discuss its user experience aspects from the trainees' and job coaches' perspectives.