• First introduced a five-keypoint geometric model for pitaya growth pose analysis. • Built the first 3D keypoint dataset of pitaya for pose analysis and harvesting. • Proposed 3DPKDNet for 3D keypoint localization and pose estimation in pitaya automatic harvesting. • Depth prediction branch and MK Transformer Block significantly improve 3D keypoint accuracy. • Achieved 94.05% PDK@0.2 and 15.08° mean angular error in pose estimation. As modern smart agriculture advances, automated fruit harvesting has emerged as a crucial technology for improving labour efficiency and maintaining yields.However, harvesting robots confront major visual perception issues, especially when dealing with crops that require precise harvesting techniques and operating in unstructured orchards with environmental variable and occlusion. This study proposes a geometric model based on five keypoints (two cutting points and three posture points) and constructs a corresponding 3D keypoint dataset for pitaya. Based on this model, a CNN-based 3D Pitaya Keypoint Detection Network (3DPKDNet) is built to estimate both the 3D coordinates of the cutting points and the 3D pose vector of the fruit. The network features a 3D Keypoint Head that integrates a 2D heatmap branch with a depth prediction branch. The model improves depth convergence and 3D keypoint localization accuracy by improving heatmap, depth, and pose vector losses simultaneously. To enhance robustness under complex backgrounds and occlusion, this study introduces the Mask-Keypoint attention module (MK Transformer Block), which uses mask-guided attention to modify both keypoint and depth predictions. The experimental results demonstrate that 3DPKDNet achieves a 2D keypoint detection mAP of 73.63%. Furthermore, the 3D keypoint detection rate reaches 94.05% at a threshold of 0.2, with an average pose vector error of 15.08° across varying distances. The proposed dataset and algorithm provide a reliable technical foundation for the robotic harvesting of stem-cutting crops in orchard settings.
Yi et al. (Sun,) studied this question.