Development of a Human–Robot Hybrid Intelligent System Based on Brain Teleoperation and Deep Learning SLAM

To achieve the better navigation performance of a mobile robot in the unknown environments, a novel human-robot hybrid system incorporating a motor-imagery (MI)-based brain teleoperation control is presented in this paper, where a deep-learning-based active perception is developed in the simultaneous localization and mapping (SLAM) framework. Using the deep-learning-based object recognition in the red-green-blue-depth (RGB-D) data acquisition process, the designed SLAM approach can select the valid feature points effectively, and the speed of displacement tracking can be improved by combining the oriented FAST and rotated BRIEF (ORB) SLAM algorithm with the optical flow method. The global trajectory map can also be mended using graph-based nonlinear error optimization. In addition, to build the connection between human intentions and the robot control commands flexibly in the developed mobile robot, a common spatial pattern (CSP)-based support vector machine (SVM) classification algorithm is proposed so that the control commands can be obtained directly from the human electroencephalograph (EEG) signals, which are preanalyzed and classified using the phenomena of event-related synchronization/desynchronization (ERS/ERD). Experiments involving several operators have verified the effectiveness of the proposed framework in the actual unstructured environments.