Answers are grounded in this paper and cite their sources.
Shows tool divergence confirmation in programming languages, indicating needs for better development practices.
An agent that discovers a tool at runtime learns what it does from its documentation, and documentation drifts. We show this gap is not closable by scale: confirming that an implementation diverges from its documentation requires execution (Rice's theorem forecloses any complete static method), while reasoning over (specification, weights) yields only suspicion. We ground the problem in sixteen real, silent divergences reproduced live across ten Python/JS packages, then measure suspicion and confirmation across a 0.5B-7B scale ladder using injected divergences immune to training-set recall, with clean twins so suspicion carries a false-positive rate. No model confirms a divergence it has not executed (0/12 at every rung); suspicion appears only when the documentation names the hazard (92% vs 25% at the governing rung, Fisher's exact p=0.0014) and is identical on honest and diverging tools documented alike (no column effect, p=1.0). We then build and run the missing stage: a behavioral admission gate that derives a contract from documentation, executes the tool in a sandbox, and admits it only against observed behavior - it rejects six of seven divergences the model confirmed none of, admits all five clean twins, at about 0.13 s per tool; the one miss names no checkable property in its documentation. Targets were pre-registered. No frontier model was run.
No takes yet. Share an insight, caveat, or question.
Abhijeet Verma (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: