Boundary-weighted supervision and multi-scale context enhancement for greenhouse tomato instance segmentation using YOLO11n-seg

Accurate tomato instance segmentation in greenhouse scenes remains challenging because of fruit occlusion, instance contact, ambiguous boundaries, and the visual similarity between green fruits and leaves. This study presents a lightweight YOLO11n-seg framework that combines a morphology-based shape-boundary-driven (SBD) weighting strategy with the previously proposed CMPRD multi-scale context module. SBD constructs a narrow contour band for each ground-truth instance and increases the contribution of boundary pixels to the mask loss during training without altering the inference graph, whereas CMPRD replaces SPPF near the end of the YOLO11n-seg backbone to enhance multi-scale contextual representation. Across five independent runs on the 128-image validation split (mean ± standard deviation), the combined model increases Mask Recall from \(0.7656± 0.0046\) to \(0.7726± 0.0039\) and Mask mAP50–95 from \(0.6587± 0.0025\) to \(0.6812± 0.0026\) relative to YOLO11n-seg, while Mask Precision changes from \(0.7989± 0.0037\) to \(0.7961± 0.0035\) . In the fixed seed-0 comparison with additional baselines, the larger YOLOv8s-seg and YOLO11s-seg models achieve Mask mAP50–95 values of 0.69161 and 0.69434, respectively, whereas a literature-guided reimplementation of Enhanced YOLO11n-Seg achieves 0.67840 under a comparable lightweight computational budget. On the untouched 161-image official test set, the primary Mask mAP50–95 results, reported as the mean ± standard deviation across all five runs rather than the best-performing run, are \(0.6529± 0.0016\) for YOLO11n-seg and \(0.6782± 0.0016\) for the proposed model, corresponding to a mean gain of 0.0252 and a paired 95% confidence interval of [0.0227, 0.0277]. Among successfully matched instances with mask IoU \(≥ 0.5\) , Boundary IoU increases from \(0.612± 0.004\) to \(0.651± 0.003\) , and Boundary F-score increases from \(0.744± 0.003\) to \(0.779± 0.002\) . On the same validation split and seed-0 development run, a parameter/FLOP-matched convolutional control achieves a Mask mAP50–95 of 0.66530, compared with 0.67007 for CMPRD. The effects of spatial boundary weighting and overall mask-loss scaling are not isolated, and cross-dataset generalization remains unverified.