Research and trials on multi-pose picking of Sichuan pepper based on multi-task perception

Compared to other fruits, Sichuan pepper is difficult to pick mechanically due to its long, sprawling branches, irregular growth pattern, prickly bark, and small fruit size. As a result, picking is currently carried out entirely manually. This study addresses the challenges of low efficiency and high fruit damage encountered in Sichuan pepper picking robots. Aiming to achieve systematic and full-process picking, we propose a novel Sichuan pepper picking robot. The main contribution of this paper is to present a complete hardware-software collaborative safe picking scheme. The specific research process is as follows: First, high-precision recognition of Sichuan pepper and branches is achieved using the multi-task model YOLOP2, laying a reliable visual perception foundation for subsequent operations. Next, a spatial pose estimation method for Sichuan pepper is proposed by combining segmented point clouds and Principal Component Analysis (PCA), solving the core difficulty of quantifying poses in unstructured environments. Then, by integrating collision detection with a picking orientation model, a multi-pose adaptive picking decision strategy is constructed to determine the optimal picking pose and feed parameters that avoid branch interference. Finally, a flexible swallowing-type end-effector matched to the picking method is developed, which significantly enhances the fault tolerance of visual information against disturbances in unstructured scenes from the mechanism design perspective. The results show that the recognition performance metrics of Sichuan pepper and branches achieve an mAP@0.5 of 0.9101 and an mIoU of 0.8883, respectively. Meanwhile, the fitting error of the pepper pose axis is investigated from different viewing angles. The study reveals that the middle axis (e₂) error under the parallel view is the smallest, with a bias of 0.5 mm and a MAPE of 16.74%. A total of 30 groups of Sichuan pepper field picking trials are conducted on the Sichuan pepper picking robot. The overall average picking net rate is 50.2%, and the picking success rate is 56.7%. These experimental results verify the preliminary feasibility of the proposed integrated scheme and provide a valuable technical reference for the subsequent research, development, and engineering exploration of adaptive picking robots for Sichuan pepper in unstructured environments.