A real-scene-oriented design method for human–computer interaction with mid-air gestures: taking an in-vehicle gesture interaction scene as an example

With advances in sensors and recognition algorithms, mid-air gesture interaction has has received extensive attention. The user-elicitation method is mainstream for its naturalness and intuitiveness, but it cannot avoid the Midas Touch problem from unconscious background gestures. This paper proposes a real-scene-oriented method for human-computer interaction mid-air gesture design to optimise the user-elicitation gesture design process. This method adds the background gesture collection and overlapping gesture filtering process. A set of background gestures was recorded from a real scene and compared with the user-designed gesture set generated through user-elicitation design processes to determine the optimal gesture set for the real scene. The DigCode structured gesture encoding system was used to record and store gestures for subsequent comparison.To validate the feasibility of the proposed method, it was implemented in an in-vehicle interaction scene. The results verify the significant conflicts between user-designed gestures and background gestures and demonstrate that the proposed method improves the usability of mid-air gesture interaction. The method effectively addresses limitations of traditional user-elicitation methods and offers solutions for mitigating the Midas Touch problem in real-time monitoring-dependent interaction modalities such as voice control and motion sensing systems.