Fathom v25 / styxx v7.7.9: The Gauntlet that Catches Itself — pre-registered AI evaluation infrastructure with in-production bar-weakness detection

Headline: the gauntlet catches itself. Two consecutive bar-weakness discoveries in a single session — D3 length-control caught by accident (Baseline-007 unexpected PASS), D4 capitalization-control caught by deliberate systematic confound audit — with regression-tested fixes, three pre-registered detection submissions tested under the strengthened v3 bars, and a 3584-word preprint documenting the recursion-of-discipline pattern. What's new in v25 (styxx 7. 7. 9) vs v24 (styxx 7. 7. 7): D3 length-control bar (7. 7. 8) — detector AUC must beat the length-only oracle by ≥ 0. 10 on both partitions. Shipped six minutes after Baseline-007's accidental v1 PASS exposed the length artifact in expectedconsensus (truth ~3. 9w, folklore ~7. 5w). D4 capitalization-control bar (7. 7. 9) — detector AUC must beat the cap-ratio oracle's absolute AUC by ≥ 0. 10 on both partitions. Discovered by auditconfounds (): cap-ratio is INVERTED on this corpus (truth has higher proper-noun density because canonical answers like "Paris", "Newton" are mostly proper nouns). auditconfounds () primitive — scans 8 surface features as oracle-detectors, reports direction-agnostic AUC + Spearman ρ to length. The structural counterpart to D3, available via styxx gauntlet-audit-confounds. Three pre-registered detection submissions under v3 bars: Baseline-008 (embedding similarity): 3/4 — pre-stated 60% modal "pass D1+D2 fail D3" validated. Baseline-009 (length-residualized embedding): 1/4 — pre-stated 30% "D2 only" modal validated; n=1 demonstration that residualization removes signal rather than escaping the artifact. Baseline-010 (NLI cross-encoder entailment): 0/4 — pre-stated 20% "total fail" band validated; direction-of-effect FALSIFIED (eighth in-session falsification). Recursive-discipline preprint (PAPERᵣecursivediscipline₂026₀5₂7. md, 3584 words) — arXiv-submittable preprint arguing the contribution is the recursion (bars revising under empirical pressure), not the bars or the methods. Eight in-session falsifications documented in four paper-grade FINDINGs at papers/agent-self-audit/. Calibration record (across 5 pre-stated artifacts): Outcome-band predictions: well-calibrated (every pre-stated submission landed in its predicted modal region) AUC-range predictions: roughly two-thirds reliable Direction-of-effect predictions: SYSTEMATICALLY worst — two of the eight in-session falsifications were direction misses on this same domain (cap-ratio confound, NLI entailment). The discipline lesson is now durable: always include |AUC − 0. 5| as the prediction range; treat direction as a separate sub-prediction. The detection frontier under v3 bars (what's empirically been ruled out): Surface-form lexical (token overlap) — fails D3 (tracks length too closely) Classical NLP (TF-IDF) — fails K1/K3 on classification (folklore lacks lexical signature) Raw semantic embedding similarity — fails D3 (structurally equivalent to length on D1) Length-corrected embedding — fails D1+D3+D4 (residualization subtracts signal) Pre-trained NLI cross-encoder — fails all four (MNLI training doesn't transfer to factual-restatement detection) Methods still untested under v3 bars: cross-vendor consensus disagreement, perplexity against a calibrated prior LM, knowledge-graph lookup, fine-tuned classifier on (q, r) pairs. All operator-territory at present. Methodological contribution: We argue that the moat of disciplined AI evaluation is not the bars or the benchmark but the recursive pattern: bars revise under empirical pressure from real submissions; the discipline of pre-registration BEFORE each run makes wrongness visible rather than hidden; the infrastructure improves itself on a session timescale. The asymmetry is intentional: submitters may run their methods privately, only publishing their best results; we publish every gauntlet run we make against ourselves, every prediction we lock, every falsification that follows. Over time, this compounds — the floor accumulates rigor faster than any single submission can erode. Reproduce: pip install styxx==7. 7. 9 styxx gauntlet --method styxx. gauntlet: ₘajoritybaselineₚredict --task classification styxx gauntlet-audit-confounds styxx leaderboard --rows-only Source: github. com/fathom-lab/styxx · Tag: v7. 7. 9 · PyPI: pypi. org/project/styxx/7. 7. 9 · Recursive-discipline preprint: PAPERᵣecursivediscipline₂026₀5₂7. md Predecessor: v24 (10. 5281/zenodo. 20418532) — Fathom v24 / styxx v7. 7. 7, the seven-method empirical floor with the closed-loop self-audit demonstration. v25 advances by shipping the bars-catching-themselves recursion (D3 by accident, D4 by deliberate scan), three pre-registered baselines under v3 bars (all modal-validated, none cleared), and the recursive-discipline preprint. License: CC-BY-4. 0 (this deposit, paper, data, findings, leaderboard). MIT (styxx Python code). Notes: Drafted with assistance from Claude Opus 4. 7 (1M context) during a continuous 2026-05-27 session. The recursive-discipline preprint §5 specifically describes how this paper itself caught a prediction error in its own draft text (the cap-ratio direction was originally described as "predicted in the positive direction" before correction). The methodological recursion the paper proposes — pre-stated gauntlets that catch their own bar weaknesses in production — is the contribution we expect the field to either replicate or refute within a year.