Towards virtual reality-based telemanipulation systems for military applications

Robot manipulation has been on the rise in the past decade. This has yielded tremendous growth in low-cost commercial off-the-shelf robot arms. Employing such arms on the ground, aerial vehicles, and space robots is called mobile manipulation, and it is an active research topic in robotics. The U.S. Department of Defense (DoD) has been interested in using mobile manipulation for military applications like intelligence, surveillance, and reconnaissance (ISR) missions. The idea is to replace soldiers with mobile manipulating robots to complete essential tasks in hazardous areas like unexploded ordnance (UXO). One of the challenges in mobile manipulation is to complete the desired tasks beyond the operator's line of sight. Immersive technologies like VR/AR could address this challenge. VR/AR provides immersive situational awareness to the operator in the remote site. It also maps human body motions to the robot arm. Integrating such technologies enables the operator to leverage his experience to complete manipulation tasks through mobile robots, called telemanipulation. This paper presents hardware/software design and experiment results of telemanipulation.