A Progressive Hybrid Automatic Switching Visual Servoing Method for Apple-Picking Robots

Position-Based Visual Servoing (PBVS) and Image-Based Visual Servoing (IBVS) struggle to balance end effector pose accuracy and robustness in apple picking. They are also prone to target loss and control singularities. A progressive Hybrid Automatic Switching Visual Servoing (HAVS) method is proposed and applied to an apple-picking robotic system. HAVS integrates PBVS and IBVS to coordinate control of the manipulator end effector pose. A depth-based switching function is designed. When target depth is below an optimal threshold, the controller switches to PBVS for precise final positioning. This reduces target loss and control singularities. An adaptive proportional-derivative (PD) controller with fuzzy gain scheduling updates the control gains online to enhance responsiveness and stability. The hardware consists of a six-axis manipulator, a depth camera, and a mobile base. You Only Look Once version 5 (YOLOv5) performs apple detection and generates control commands. Indoors, success rate was 96%, which was 4 and 10 percentage points higher than PBVS only and IBVS only. Average picking time was 12.5 s, 0.3 s, and 1.1 s shorter. Outdoors, success rate was 87.5%, average time was 13.2 s, and damage rate was 4.2%. This method provides a reference implementation for visual servo control in agricultural picking robots.