Exploring a Humanized Multimodal Interaction Design for Medical Terminal: A Fusion Scheme Based on Graphical User Interface, Voice Output and Eye Control Technology

This study explores how multimodal interaction can improve medical terminal that currently rely on touch interaction. The design was grounded in two weeks of on-site observation across five tertiary hospitals and 12 semi-structured interviews, ensuring support for hands-busy patients and other special-needs users. We developed a high-fidelity prototype that combines concise voice output prompts for feedback and eye control for selection, while preserving a touch-only baseline and an eye-controlled-only variant. A between-subjects experiment (N = 69) compared the three interaction modes in a controlled, noise-rich mock lab. Relative to traditional touch, the eye-controlled + voice system reduced perceived mental demand, effort and frustration, and raised subjective usability and technology acceptance. Task times were 13–24% longer because of dwell-based selection, but users judged the hands-free benefit to outweigh this cost, especially when carrying medical documents. The results confirm Multiple-Resource and Cognitive-Load predictions: distributing information across visual and auditory channels lightens cognitive burden and stabilizes emotional arousal. Designers should therefore (1) add clear voice feedback to any eye-controlled interface, (2) minimize dwell delays through adaptive timing, and (3) provide brief in-situ tutorials for first-time users. Overall, multimodal medical terminal could make digital hospital services more inclusive without sacrificing learnability.