Multi-Touch Gesture of Mobile Auditory Device for Visually Impaired Users

In this study, we conducted experiments with 15 visually impaired users for graph reproduction tasks with additional modalities. The question we have tried to answer is the possibility of integrating multi-touch gestures into auditory graphs as an interactive tool for mental visualization and understanding data. We introduce a new multimodal approach, based on multi-touch gesture interaction, aiming to have more accurate mental models of the plots and to improve the accessibility of smartphone's user interfaces. The findings contribute to confirm empirically that multi-touch gestures provide more precise outcomes than passively listening to the audio. The evaluation confirms that the implementation of additional modalities to auditory graphs could improve the perception of the mental model of the plot.