Deep learning-based multi-view perception and adaptive grasp pose correction for cooperative dual-arm fruit harvesting under occlusion

This study presents a failure-recovery cooperative harvesting pipeline for a dual-arm agricultural robot, where each arm harvests independently and switches to cooperative manipulation only when a single-view grasp on an occluded fruit fails. Such cooperation is necessary because accurate grasp pose inference degrades substantially when target fruits are heavily occluded by surrounding foliage or branches; existing approaches, including explicit 6D pose estimation and shape-completion-based methods, depend on reconstructing geometric information that is often unavailable under heavy occlusion. To address this limitation, the proposed framework, referred to as the Cooperative Active Harvesting Robot Scheme (Co-AHRS), integrates direct grasp pose inference from incomplete point clouds, active multi-view perception, and closed-loop visual servoing. Instead of reconstructing hidden geometry, the framework leverages Contact-GraspNet to generate feasible grasp candidates directly from partial observations. When the initial grasp attempt fails—due to either inaccurate pose inference under heavy occlusion or physical displacement of the target fruit during approach—the opposite arm is reassigned as an auxiliary observer whose upward-facing viewpoint enables closed-loop visual servoing, continuously estimating the 3D positional error between the gripper and the target fruit for real-time pose correction. The Co-AHRS was evaluated through a laboratory implementation and achieved harvest success rates of 89.2% and 78.5% under unoccluded and occluded conditions, respectively. The proposed framework also substantially outperformed frustum-based localization (75.0% and 55.2%) and explicit 6D pose estimation (57.1% and 7.7%). A further evaluation on a second configuration built from two same-model manipulators verified that the two arms harvest in parallel and quantified the occluded condition by a visible surface ratio of 0.549 ± 0.101. These results demonstrate that direct grasp inference combined with cooperative multi-view visual feedback is an effective strategy for autonomous fruit harvesting in occlusion-prone orchard environments.