VibeGuardBench: A Deterministic Benchmark for Repair-and-Regression in AI-Generated Web Applications

Prompt-driven AI application builders can produce convincing full-stack web applications from a single specification. Existing benchmarks score that first act: what the builder generates. What happens next — when deterministic engineering feedback identifies concrete defects and the builder is asked to fix them — is unmeasured. VibeGuardBench is an open benchmark for that second act. Three frozen application specifications (appointment booking, multi-role dashboard, ordering/inventory) are generated by commercial AI builders; a deterministic 39-check oracle suite (build, types, lint, functional, authorization, security, accessibility, persistence) measures each application; machine-generated feedback is delivered per failing check; and the entire suite re-executes after every repair, for up to three iterations. The benchmark reports repair success rate, regression rate among previously passing checks, and net quality change, with raw counts and a preregistered analysis plan. A harness-validation pilot (one application, excluded from research claims) exercised the full loop and surfaced fifteen harness defects before data collection.