From Weakly Supervised Segmentation to Diagnostic Decisions: An Annotation-Economics Review for Medical Imaging

Accurate medical image segmentation and the diagnostic decisions it supports are increasingly constrained by the cost of supervision: dense voxel-level annotation by expert clinicians remains slow, expensive, and difficult to scale, leaving many anatomies, pathologies, and modalities sparsely labeled. A large literature therefore relaxes the demand for full masks, yet existing surveys often catalog broad forms of limited supervision without directly comparing the practical trade-off between label granularity, annotation burden, and segmentation quality. This review focuses on weakly and coarsely labeled settings (image-level tags, points/clicks, bounding boxes, scribbles, and coarse masks, together with gaze or partial annotations as separate, protocol-dependent forms of supervision) and organizes them through an annotation-economics perspective that considers per-case annotation cost, required expertise, scalability, and the reported performance gap relative to full supervision. The evidence suggests that box and scribble supervision can provide favorable cost–performance trade-offs in several medical segmentation studies, although the reported metrics and datasets are heterogeneous and should not be interpreted as a unified quantitative frontier. We next examine the foundation-model turn, namely the Segment Anything Model and its medical derivatives, and argue that promptable models re-encode rather than eliminate human supervision, moving part of the effort from offline labeling to inference-time interaction. Finally, we extend the analysis beyond Dice to ask whether weakly produced masks support downstream clinical tasks: aggregate measurements may be relatively tolerant of moderate boundary error, whereas texture and high-order radiomic features can be substantially more sensitive. We close by identifying benchmark fragmentation, annotation variability, and the still-limited evidence connecting weak segmentation to clinical outcomes as open problems and propose a unified “weak label → task” benchmark for future evaluation.