Empirical audit reveals persistent byte-false claims across five leading AI model families over 31 rounds, highlighting systematic defect migration in structural decision replay.
Empirical results from 31 audit rounds on the XZ Protocol, a construct-validation preregistration for structural-decision replay of the XZ Utils supply-chain compromise (CVE-2024-3094). The audit program produced 199 seat reports from five core vendor families (Anthropic, OpenAI, Google, Moonshot, Zhipu), yielding 832 structured findings (136 Class A byte-false claims). Primary result: 25 consecutive rounds (R7--R31) with Class A present, with 102 extracted Class A findings across 19 rounds and description-level or report-metadata evidence for 5 additional rounds (1 round ASSERTED from the companion Heartbeat paper). Secondary result: defect migration trajectory with recurrence across code/CI (41 findings), cross-layer consistency (23), methodology/dispatch (11), provenance/amendment (6), and unclassified (21) locus layers. The deposit includes the paper, per-round recurrence witness table, per-finding migration classification, the classification script, and the structured corpus data. Four audit rounds across three non-producer model families; carrying seat returned CONDITIONAL PASS at R4. This paper makes no effectiveness claim about any component of the Structural Honesty program.
No takes yet. Share an insight, caveat, or question.
Bilal Syed Arfeen (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: