The Impact of Embodiment and Output Modality on Learning with Conversational Agents in VR

In Virtual Reality (VR) learning applications, the integration of pedagogical agents is particularly promising, as they function as a virtual social support. With the recent developments in Large Language Models (LLMs), conversational language models are now available that enable natural language interaction in a VR environment. We evaluated four possible designs for LLM-based agents (n = 21) in an aircraft-engine training. Focusing on the primary questions when bringing LLM-based agents into VR: whether to use text or speech and whether to embody the conversational agent in 3D or not. Pre-and post-tests were used to measure retention and questionnaires were used to measure the User Experience (UX) of different design variants. Contrary to our hypotheses, retention was higher when using a non-embodied design compared to an embodied design. The output modality did not have a significant impact on the learning success, but it did have an impact on the UX.