When Execution Feedback Reveals the Expected Output: A Forensic Study of Answer Leakage in Agentic Code Repair

Coding agents are often given test-runner feedback and then credited with improving a repair. This preprint audits three experimental designs from our own research programme and finds no validated clean positive evidence for the proposed benefit of controller-owned verification. In the primary 1.5B QuixBugs arm, a row-bound strict recount found that 156 of 169 consumed feedback messages (92.3 per cent) rendered the fixed expected-output field. The broader directory-glob audit reported 191/206 keyword hits, whereas retained rows give 191/203 keyword hits across the 1.5B and 3B tiers, of which 189 are strict field matches. In HumanEvalFix, a provenance-bound matcher proved that at least 316 of 417 out-of-band feedback rounds (75.8 per cent) exposed the task-specific rendered assertion; 289 of 417 (69.3 per cent) exposed an explicit equality against an expected value. An earlier design emitted raw pytest output but was not scanned and remains unresolved. The pre-registered QuixBugs invalidation rule fired when a nominally inert stale control beat the blind baseline. The registered cell-level McNemar exact test gave p = 0.0018. A post-campaign clustering-honest analysis corroborated the decision with a task-level sign test, p = 0.0117, and a paired task-clustered bootstrap interval of [0.046, 0.241]. Two post-campaign exploratory arms were time-stamped before their own execution. The rich-feedback out-of-band arm resolved 33 of 87 task-seed cells; a pass-or-fail-only out-of-band arm resolved 21 of 87, and a scaffold-only arm resolved 21 of 87, against 22 of 87 for the blind baseline. The rich-feedback versus pass-or-fail contrast was +0.138, with a paired task-level sign-test p = 0.0020 and a task-clustered bootstrap 95 per cent interval of [0.069, 0.230]. The two clean arms showed no detectable gain over baseline at the achieved precision; this is not an equivalence result. A later 3B run overwrote 783 row-referenced HumanEvalFix transcript files. Exact historical leakage rates are consequently unrecoverable and are reported as conservative lower bounds. The methodological conclusion is that feedback which exposes a visible-test assertion is causally confounded with verification. Claims of verification benefit require pass-or-fail-only feedback, a same-candidate-pool selection control, task-clustered inference, and immutable transcript-to-row hashes. Record status: Author preprint, version 1.0.0. This manuscript has not been peer reviewed. Evidence availability: This record contains the manuscript. The underlying V4 evidence repository remains private and is not part of this deposit. The HumanEvalFix V3 corpus has a documented transcript-provenance failure, and V1 remains unscanned. This record must therefore not be described as a public, complete reproducibility artefact. Previous record: This work critically reassesses and supersedes the scientific claims of the earlier withdrawn CodeMC AURORA-R preprint, DOI 10.5281/zenodo.21040811.