Better Teaming Through Visual Cues: How Projecting Imagery in a Workspace Can Improve Human-Robot Collaboration

In this article, we present a communication paradigm using a context-aware mixed-reality approach for instructing human workers when collaborating with robots. The main objective is to utilize the physical work environment as a canvas to communicate task-related instructions and robot intentions in the form of visual cues. A vision-based object-tracking algorithm is used to precisely determine the pose and state of physical objects in and around the workspace. A projection-mapping technique is employed to overlay visual cues on the tracked objects and the workspace. Simultaneous tracking and projection onto objects enable the system to provide just-in-time instructions for carrying out a procedural task.