Evaluation of Concept-Probe Faithfulness Audits for Computer Vision Agriculture Disease Classifiers Under computed-to-Field Domain Shift: An Interpretability Validity Bound for Smallholder Fungicide Triage

Concept-probe audits are increasingly proposed as a way to certify that a computer vision disease classifier is attending to the lesions it claims to detect rather than to spurious correlates of the training distribution. For smallholder fungicide triage, where a false negative can cost a season's harvest and a false positive can waste scarce cash on an unnecessary spray, the audit is only useful if its verdict survives the computed-to-field domain shift that separates curated training imagery from the photographs farmers actually submit. the present analysis formalises the concept-probe faithfulness audit as a statistical test on the alignment between probe direction and model evidence, and derives an interpretability validity bound: the probability that a classifier passing the audit in the computed domain also passes it in the field domain, expressed as a function of the probe–evidence alignment gap and the domain-shift magnitude. The bound is evaluated through a replication-based design that characterises the operating characteristics of the before-after-control-impact estimator under a declared variance structure (between-cluster SD = 6.0, residual SD = 4.5, intraclass correlation = 0.64). Power was estimated across 2000 design cells spanning cluster counts of 8, 16, 24 and 32 and effect sizes of 3, 5 and 8 percentage points. The analysis indicates that 52 clusters are required to detect a 5-percentage-point shift at 80% power, and that the audit's field-domain pass probability falls below 0.5 once the alignment gap exceeds roughly one third of the probe's computed-domain margin. The contribution is a validity bound that lets a triage programme state, before deployment, the confidence it can place in a concept-probe certificate.