Answers are grounded in this paper and cite their sources.
Explanations of machine learning models are read by people but rarely evaluated with them, and automatic metrics score attribution vectors rather than what a reader receives. We operationalise the horizon, the hermeneutic circle, and the hermeneutics of suspicion as the Hermeneutic Circle Test (HCT), a human-free screen for explanations of tabular classifiers. HCT checks whether an explanation’s evidence lies inside a calibrated data horizon, whether its sentences survive on-horizon interventions, whether its account of a case agrees with the whole model and at which scale, and whether it helps machine readers learn the model. Over seven explainers, four datasets, and two model families (mean ± standard deviation over eight settings), the best explainer, KernelSHAP, verified 0.65 ± 0.15 of its claims, while random attributions verified 0.22 ± 0.08. On the same explanations, established faithfulness, remove-and-retrain, and disagreement metrics missed impossible evidence and narratives contradicting their own attributions, and sign-blind metrics scored sign-flipped explanations like the originals. Claim tests withstood scaffold attacks that adapt to the horizon and to the test itself; a scaffold that memorises the audited records weakened them and deceived every explainer, but part–whole agreement still flagged it. HCT is a reproducible screen to apply before user studies.
No takes yet. Share an insight, caveat, or question.
Şadi Evren Şeker (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: