Multitasking across Physical and Virtual Displays: the Effect of Spatial Discontinuity and Task Load

Optical see-through head-mounted displays (OST-HMDs) allow users to interact with both physical and virtual displays overlaid in the real world within augmented reality (AR) environments. When arranging these displays in space, spatial discontinuity-such as differences in viewing distance or lateral separation in the visual field-may occur. This study examined the effects of displays' spatial discontinuities on multitasking performance and perceived workload in AR, as well as how these effects were influenced by task load. Participants completed a multitasking simulation with four tasks distributed across physical and virtual displays under four display configurations: two configurations with spatial discontinuity, one cross-display configuration without discontinuity, and a baseline configuration where all tasks were shown on the physical display. Each configuration was tested under two task load levels. Results showed that OST-HMD-supported AR significantly impaired multitasking performance, regardless of whether spatial discontinuity between virtual and physical displays was present. Introducing display discontinuities through small depth separations or alignment offsets produced no additional significant influence on performance. The AR configurations also shown increased perceived workload than the baseline, but spatial discontinuity did not significantly affect workload ratings. Task load modulated the effect of display configurations on performance as well as physical and frustration ratings of the perceived workload.