Supporting Bi-Directional Collaboration with Unmodified Immersive Applications Using AccompliceVR

With the increasing popularity of virtual reality (VR) applications, allowing interaction with non-immersed spectators has become an important supporting functionality. To this end, a variety of VR streaming systems focus on allowing remote spectating experiences. However, such systems lack mechanisms that allow spectators to assist or explore inside VR applications alongside the VR players. To address, we explore AccompliceVR (AVR), a novel streaming and remote collaboration system. Using built-in OpenVR functionality, AVR introduces avatar representations of spectators overlaid into the normal stereo views of unmodified VR applications. AVR also enhances the remote spectator experience by stitching frames of video from the VR player to reconstruct a pseudo-panoramic view. This allows them to move around and point at the environment with an avatar. We first evaluated that AVR is compatible with a selection of the most popular VR games. The system is then validated through a paired user study with 30 participants using a mixed-design pattern. Participants played two well-known VR games (as 'pilot') and were assisted by their partner in the pair (as 'copilot'). The results support that AVR shows significant improvement in communication between pilots and copilots and enhances the pilot's understanding of instructions from their copilot.