Concept-probe causal sufficiency audits ask whether a vision classifier's internal representation of a disease concept is sufficient to reproduce the classifier's own decisions, and they are increasingly proposed as a gate before a model is trusted to triage fungicide applications. Under sparse field-scouting ground truth, however, the audit's verdict is not a property of the model alone: it is jointly determined by the probe's identification power and by the sampling design that produced the labels against which sufficiency is judged. the present analysis formalises that dependence. A concept-probe sufficiency audit is written as a before-after-control-impact comparison of probe-implied decisions against classifier decisions, and its operating characteristics are derived under a declared variance structure with an intraclass correlation of 0.64, a between-cluster standard deviation of 6.0 and a residual standard deviation of 4.5. Power was estimated across 2000 replications per design cell for cluster counts of 8, 16, 24 and 32 and effects of 3, 5 and 8 percentage points. The analysis yields an interpretability identification bound: 54 clusters are required to detect a 5-percentage-point sufficiency gap at 80% power, a requirement that exceeds every cluster count examined and that most field-scouting programmes cannot meet. The bound is presented as a design constraint on audit claims, not as a verdict on any particular classifier, and it implies that sufficiency audits under sparse scouting should report the minimum detectable gap alongside the audit outcome.
No takes yet. Share an insight, caveat, or question.
Amégan et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: