E-YOLOv11-RGBD perception-to-action framework for multicrop greenhouse robotic harvesting

Selective robotic harvesting requires image detections to be converted into spatially valid, reachable, graspable, and executable robot targets. This study presents E-YOLOv11-RGBD, a YOLOv11-based red-green-blue plus depth (RGB-D) perception-to-action framework for tomato, strawberry, and pepper harvesting. The detector adapts the medium-sized YOLOv11 model (YOLOv11-M) with a P2 small-target branch, bidirectional multiscale fusion, C2PSA-iEMA attention refinement, Wise-PIoU-based bounding-box regression, hard-negative learning, scale-aware training, and validation-derived confidence selection. Accepted RGB detections are associated with aligned depth from an Intel RealSense D435i, deprojected into three-dimensional (3D) coordinates, transformed to the UR3e base frame, compensated for Tool Center Point (TCP) geometry, and screened for inverse-kinematics feasibility and gripper compatibility before Real-Time Data Exchange (RTDE) command generation. On the fixed held-out detector test split, YOLOv11-M produced precision, recall, mAP @ 0.5, and mAP @ 0.5-0.95 values of 95.0%, 93.0%, 96.5%, and 90.0%, respectively, while E-YOLOv11 produced 97.4%, 95.8%, 98.3%, and 93.1%. These are single-training-run point estimates and do not establish between-seed statistical superiority. In the robotic comparison, the observed harvesting-success proportion was 51.7% (62/120) for the baseline configuration and 92.5% (111/120) for the enhanced configuration. This system-level difference reflects the combined detector and target-validation changes and is not attributed to the detector alone. Independent RGB-D localization testing yielded a mean 3D Euclidean localization error of 10.20 ± 3.74 mm. Each system configuration comprised 120 crop-level harvesting attempts, corresponding to 40 attempts per crop class and 240 physical harvesting attempts overall. The harvesting trials used the UR3e-D435i-2FG7 subsystem in a stationary laboratory configuration at an approximately 45 cm initial camera-to-scene distance under controlled illumination. The LD-60 mobile platform was evaluated separately at predefined positions; autonomous mobile navigation was not part of the harvesting experiments. The reported results therefore apply to the evaluated proof-of-concept conditions and should not be interpreted as validation of autonomous mobile harvesting or cross-greenhouse generalization.