Foundation Model‐Assisted Segmentation Enables Robust Field‐Based Severity Estimation: A Case‐Study of Coffee Leaf Rust

ABSTRACT Accurate estimation of coffee leaf rust ( Hemileia vastatrix ) severity remains challenging due to symptom heterogeneity, illumination variability and observer subjectivity, particularly under field conditions. We developed a segmentation pipeline combining zero‐shot foundational models (SAM3) and fine‐tuned deep learning (SAM2) for estimation of percentage severity and compared them with three other segmentation approaches: DeepLabV3+ (convolutional neural network model), ImageJ (colour thresholding) and the pliman R package (palette‐based). A total of 1285 images were collected across several field plots, and 606 pixel‐level rust lesion masks were curated for model fine‐tuning and independent evaluation. A two‐stage detection–segmentation workflow integrating YOLOv8 and SAM2 enabled robust leaf extraction from complex branch‐level images, with detection performance consistently exceeding 0.98 across metrics. For lesion segmentation, at the pixel level, the zero‐shot foundation model SAM3 achieved the highest disease segmentation performance (Dice ≈0.91; IoU ≈0.83), with balanced precision (≈0.93) and recall (≈0.94). DeepLabV3+ also performed well (Dice ≈0.86; IoU ≈0.75), whereas classical threshold‐based approaches showed high recall but substantially lower precision, indicating frequent false positives and producing a systematic positive bias. SAM2 exhibited more conservative performance, yielding higher precision but reduced recall relative to SAM3. At the leaf level, the highest agreement between predicted and reference severity estimates was found for SAM3, followed by DeepLabV3+ and SAM2, while classical methods overestimated diseased area. Overall, SAM3 led to a reliable, scalable and consistent severity estimation under real‐world field conditions.