Predictive Human-Machine Interface for Teleoperation of Air and Space Vehicles over Time Delay

Current plans for the exploration of Moon and Mars envision the use of telerobotic systems controlled from orbiting laboratories. The advantage of telerobotics is that it combines the resilience, endurance and precision of robots with the inherent flexibility, anticipation and decision making capabilities of humans. The primary disadvantage of telerobotics is the communication time delay in the human-robot control loop. The delay can lead to a loss of situation awareness, an increase in operator work load, and an overall decrease in effectiveness and efficiency of the human-robot system. Most of the effects of the delay can be mitigated by the use of predictive displays, presenting the operator with a simulated system state. This paper presents current work on such a predictive display designed to support an operator in remote flight and landing of space robots and unmanned aerial vehicles. The Adaptable Human-Machine Interface was developed for hardware-in-the-loop laboratory experiments with a Parrot A.R. Drone 2.0 quadcopter as test case. Based on live video from two on-board cameras, attitude and velocity telemetry, and control inceptor deflection, the interface calculates a predicted flight path and attitude and presents it in a “tunnel in the sky” display. The graphical display itself was developed in the Unity3D game engine. The paper describes the implementation of the interface between Unity3D and the A.R. Drone, the dynamic model of the quadcopter, and the prediction algorithm. The paper also discusses the results of flight tests involving a number of test subjects and projects the path forward in the development of this technology.