Empirical audit reveals persistent byte-false claims during AI evaluation of software supply-chain compromises, indicating limits in automated self-verification.
A preregistered structural-decision replay of the XZ Utils supply-chain compromise (CVE-2024-3094), built as a bounded verification instrument and audited by independent AI systems across four vendor families (Claude/Anthropic, Kimi K3/Moonshot, GPT-5.6 Sol/OpenAI, Gemini/Google) over 25 consecutive rounds (R7-R31). Every round produced at least one byte-false claim (Class A defect) at H = 1.0. The defect migrated monotonically from code to structure to provenance to automation to prose to the repair mechanism itself. The code layer has been clean since R22. The central empirical finding: the instrument cannot verify that it has no remaining byte-false claims about its own verification properties. This deposit contains the FROZEN protocol (v0.75, 75 amendment entries, 44 shipped files), 16 unmodified seat reports from R28-R31, two peer reviews, the six-string verification methodology (v7), the synthesis paper, and the complete program corpus.
No takes yet. Share an insight, caveat, or question.
Bilal Syed Arfeen (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: