What question did this study set out to answer?

This research aims to address the limitations of LLM self-review by introducing a robust binary verification framework.

May 17, 20260 citationsOpen Access

Binary Bitmask Verification vs LLM Self-Review at Service Scale: The Robustness Verification Framework

Key Points

This research aims to address the limitations of LLM self-review by introducing a robust binary verification framework.
Deployed RVF across 261 services in the ANKR substrate over four sessions.
Identified and categorized various types of bugs and violations.
Assessed performance with 19 services qualifying for formal proof.
Surfed 660 silent bugs across 175 source files.
Achieved 0 false-positive production blocks.
5 out of 5 negative-gate scenarios successfully passed.

Abstract

At service scale ≥ 200 components, interpretation-based verification fails. LLM self-review depends on the writer-agent reviewing its own output — structurally insufficient when the agent cannot perceive the defect it introduced. We present the Robustness Verification Framework (RVF), a four-layer binary-first verification stack that subordinates LLM judgment to deterministic bitmask predicates. Empirical: deployed across 261 services in the ANKR substrate over four sessions in May 2026; surfaced 660 silent bugs across 175 source files (the "botched latency bulk-edit" pattern), 11 codex overclaims, 22 R-008 port-authority violations, 2 invalid capabilitycaptured claims, and one fleet-wide gitleaks-quirk; 19 services formally proof-eligible per the 6-bit Layer C threshold; 5/5 negative-gate scenarios pass; 0 false-positive production blocks. We argue capability hallucination at scale is structural to LLM-author/LLM-reviewer systems and that binary verification is the only enforceable response. Companion to ArchMask (10. 5281/zenodo. 19417112), BitMaskOS (10. 5281/zenodo. 20047909), PRAMANA Protocol (10. 5281/zenodo. 19273330), and Vivechana SAR-006 (10. 5281/zenodo. 19456053).

问 AI

Bookmark

View Full Paper