Multimodal VR/MR Simulation with Empathic Agents for Nursing Education

In the field of healthcare education, including medical and nursing education, realistic and interactive simulations of various clinical scenes are required to mimic real-life situations, necessitating more than just a procedural experience. There has been a recent surge in demand for virtual/mixed reality (VR/MR) due to its advantages in providing diverse simulation environments. However, several challenges still remain to be addressed to advance to the next level of VR/MR-based clinical simulation. In this paper, I introduce the research that I have conducted on creating immersive clinical nursing scenes in VRIMR - based learning and training simulation systems, focusing on key elements such as multi-modal interaction, virtual agents, and learner performance assessment. Furthermore, I will discuss future research aimed at fostering collaboration in various healthcare fields, including nursing, and developing VRlMR simulation systems that enable learners to experience and learn from personalized clinical situations. These topics and questions will be further discussed at the IEEE VR 2024 Doctoral Consortium.