Smart insect-computer hybrid robots empowered with enhanced obstacle avoidance capabilities using onboard monocular camera

Abstract Insect-computer hybrid robots are receiving increasing attention as a potential alternative to small artificial robots due to their superior locomotion capabilities and low manufacturing costs. Controlling insect-computer hybrid robots to travel through terrain littered with complex obstacles of various shapes and sizes is still challenging. While insects can inherently deal with certain obstacles by using their antennae to detect and avoid obstacles, this ability is limited and can be interfered with by control signals when performing navigation tasks, ultimately leading to the robot being trapped in a specific place and having difficulty escaping. Hybrid robots need to add additional sensors to provide accurate perception and early warning of the external environment to avoid obstacles before getting trapped, ensuring smooth navigation tasks in rough terrain. However, due to insects’ tiny size and limited load capacity, hybrid robots are very limited in the sensors they can carry. A monocular camera is suitable for insect-computer hybrid robots because of its small size, low power consumption, and robust information acquisition capabilities. This paper proposes a navigation algorithm with an integrated obstacle avoidance module using a monocular camera for the insect-computer hybrid robot. The monocular cameras equipped with a monocular depth estimation algorithm based on deep learning can produce depth maps of environmental obstacles. The navigation algorithm generates control commands that can drive the hybrid robot away from obstacles according to the distribution of obstacle distances in the depth map. To ensure the performance of the monocular depth estimation model when applied to insect-computer hybrid robotics scenarios, we collected the first dataset from the viewpoint of a small robot for model training. In addition, we propose a simple but effective depth map processing method to obtain obstacle avoidance commands based on the weighted sum method. The success rate of the navigation experiment is significantly improved from 6.7% to 73.3%. Experimental results show that our navigation algorithm can detect obstacles in advance and guide the hybrid robots to avoid them before they get trapped.