Answers are grounded in this paper and cite their sources.
Class-agnostic promptable segmentation models, such as the Segment Anything Model (SAM) and its variants, are increasingly used to pre-segment images and thereby reduce the manual effort of building semantic segmentation datasets. Because their output masklets carry no semantic labels, standard segmentation metrics such as mean intersection over union (IoU) cannot be applied directly. This paper introduces a practical evaluation protocol for class-agnostic pre-segmentation models, built around post hoc class assignment and tested on a shipborne sea ice image dataset. Each masklet was assigned to the class occupying the largest share of its area in a set of reference labels, providing a consistent reference-based class assignment. Quality was quantified through class confusion metrics (a per-class Gini impurity and a dominant-class composition matrix) as well as through coverage and purity-weighted coverage metrics, complemented by a task-dependent downstream deviation metric, instantiated here for the derived total sea ice concentration. The protocol has been tested on 285 shipborne sea ice images by comparing four Otsu-thresholding variants and four SAM-family models. The SAM 2.1 models produced the purest masklets (Gini impurity as low as 0.002) but covered less of the scene, biasing the derived ice concentration by up to −11 percentage points, whereas thresholding achieved near-complete coverage in ice-water scenes but degraded severely in complex day-night scenes. The protocol yields interpretable, annotation-oriented diagnostics that standard metrics do not capture.
No takes yet. Share an insight, caveat, or question.
Panchi et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: