Exploring the Effects of Multimodal User Interfaces in Autonomous Vehicles

This short paper presents a study design created to explore how multimodal user interfaces (UIs) influence comfort, user experience, and overall well-being in fully autonomous vehicles. The study uses a motion-based driving simulator equipped with real vehicle components to provide realistic driving conditions. Participants are asked to complete two trials: a baseline trial and one with either a simple or extended multimodal UI combining auditory, tactile, and speech-based feedback. Driving scenarios include lane changes, emergency braking, V2X and V2V communication, and road surface variations. Data collection integrates physiological measures (EGG, EDA, PPG), self-assessment questionnaires on motion sickness (MSAQ) and user experience (UEQ), and performance on a reading comprehension task simulating non-driving activities. The paper provides some initial data about the participants’ demographics and their user experience with the UI, and gives an insight into the planned nest steps in terms of analysis and results application. The findings from this study will provide an insight into the potential of multimodal HMI design to enhance safety, comfort, and user acceptance in autonomous vehicles.