Virtual exoskeleton-driven uncalibrated visual servoing control for mobile robotic manipulators based on human–robot–robot cooperation

This paper presents an uncalibrated visual servoing control system based on the human–robot–robot cooperation (HRRC). In case of malfunctions of the joint sensors of a robotic manipulator, the proposed system enables the mobile robot to continue operating the manipulator to complete the task that requires careful handling. With the aid of a virtual exoskeleton, an operator may use a human–computer interaction (HCI) device to guide the malfunctioning manipulator. During the guiding process, the virtual exoskeleton serves as a connector between the HCI device and the manipulator. However, when using the HCI device to guide the virtual exoskeleton, there could be a risk of a large-residual problem at any time caused by non-uniform guiding. To solve this problem, a residual switching algorithm (RSA) has been proposed that can identify whether the residual should be calculated based on the motion characteristics of the artificial guiding, reducing the computational cost and ensuring the tracking stability. To enhance the virtual exoskeleton’s ability to drive the manipulator, a multi-joint fuzzy driving controller has been proposed, which can drive the corresponding joint of the manipulator in accordance with an offset vector between the virtual exoskeleton and the manipulator. Lastly, the guiding experiments have verified that, compared with the contrast algorithm, the proposed RSA has a better tracking performance. A peg-in-hole assembly experiment has shown that the proposed control system can assist the operator to control efficiently the robotic manipulator with malfunctioning joint sensors.