TouchView: Mid-Air Touch on Zoomable 2D View for Distant Freehand Selection on Virtual Reality User Interface

Selection is a fundamental interaction element in Virtual Reality (VR) and 3D user interfaces (UIs). Raycasting, one of the most common object selection techniques, has been known to have difficulties in selecting small or distant objects. Meanwhile, recent advancements in computer vision technology have enabled seamless vision-based hand tracking in consumer VR headsets, enhancing accessibility to freehand mid-air interaction and highlighting the need for further research in this area. This study proposed a new technique called TouchView which utilizes a virtual panel with a modern adaptation of the Through-the-lens metaphor to improve freehand selection for VR UI. TouchView enables faster and less demanding target selection by allowing direct touch interaction with the magnified object proxies reflected on the panel view. Repeated-measures ANOVA on the results of a follow-up user experiment on multitarget selection with 23 participants showed that TouchView outperformed the current market-dominating freehand raycasting technique, Hybrid Ray in terms of task performance, perceived workload, and preference. The user behavior was also analyzed to understand the underlying reasons for these improvements. The proposed technique can be used in VR UI applications to enhance the selection of distant objects, especially for cases with frequent view shifts.