Impact of automatic layout switching based on pre-stitched multi-view images on the performance and subjective experience of robot teleoperation: assistance for individuals with severe motor dysfunction

Abstract Teleoperated robots are revolutionizing remote work for individuals with severe motor dysfunction by expanding the scope of employment opportunities. However, the complexity of teleoperation often results in tasks requiring significant time to complete, and concerns have emerged regarding a decline in the quality of subjective experiences. This study introduces an assistive method that automatically switches between pre-stitched multi-view task-assistance layouts, which are designed to enhance spatial continuity, and non-stitched zoomed-in views, which are conducive to specific task processes. Therefore, this study focuses on the fundamental problem of determining whether the advantages of each layout can be effectively combined via automatic switching. We explore the challenges involved in achieving increased efficiency while enhancing the quality of subjective experiences. A key design constraint is that no additional input must be required for view switching, assuming that operators with severe motor dysfunction may have limited control over their upper limbs. This study involved 11 participants, including individuals with severe motor dysfunction, who assessed the proposed method utilizing a set of indices derived from a scenario modeling item delivery within a nursing facility. A trained assistant operator performed automatic switching to evaluate the feasibility of the approach. A fundamental concern with this method is that sudden changes in the size and position of a view within the pre-stitched multi-view interface may require the operator to re-recognize the remote environment, potentially conflicting with their intentions. However, results indicate that the proposed method enhances task performance in terms of task failure rate and completion time, potentially owing to the high consistency of the layouts before and after switching. Furthermore, the results indicate that the mental demand and frustration caused by external assistance are crucial factors for optimizing both task performance and subjective satisfaction. These findings contribute to expanding the scope of assistance modalities by switching between various types of operational support and increasing the potential for remote work utilizing teleoperated robots for diverse individuals.