Confirmatory evaluation demonstrates high verification accuracy across a 57-item benchmark corpus, establishing pre-registered performance criteria for text analysis pipelines.
PHILIA OS Phase 2 · 111th DOI Results ReportTrinity AI Research Team, Konyang University This document reports the confirmatory study results of ch7_v4_main_v30_full57__1_.py (SHA256: 8eb2a0e1…), the frozen v30 pipeline applying Gap-guided Residual Rescoring to the full 57-item corpus (L1×20 + L3×25 + Hard×12) under a pre-declared dual-condition establishment protocol. Governing pre-registration: 10.5281/zenodo.21974271 (110th DOI, v0.7)Study type: CONFIRMATORY ★ Verdict: ESTABLISHED (C1 ∧ C2) C1 — A arm Strict X (5-rater manual consensus, Guide confirmed): 53/57 = 92.98% ≥ 43/57 (75.4%) ✅ (+10 items margin) C2 — B arm automatic CONTRADICTION (expected match): 56/57 = 98.25% ≥ 52/57 (91.2%) ✅ (+4 items margin) N 4 items: L3_015 (PREMISE_CAPTURED — first in CH7 series; "October 2024" absent from retrieved evidence, LLM hallucinated date from claim), L3_022 (CORRECTIVE), L3_023 (CORRECTIVE), L3_026 (Wikipedia API 429 infrastructure failure). Within-corpus held-out generalization set (36 items, never scored during exploration): 35/36 = 97.2%.Hard-set (21 items): 18/21 = 85.7% — same count as exploration consensus, different composition. Novel observations: (1) First PREMISE_CAPTURED event in the CH7 series — automatic ENTAILMENT verdict was insufficient to detect evidence-fidelity failure; (2) Fidelity-axis failure in L3_NEW_A (year hallucination in summary); (3) 7 automatic NEUTRAL items where Strict X = Y, confirming comparator hyper-literalism as a residual constraint. Pre-registration violations: 0.Gate-0 adversarial review: Opus UNCONDITIONAL PASS (3 cycles); Pandora CODE GATE-0 PASS; Lucas CONDITIONAL PASS (incorporated). Corpus SHA256: 5fd61f8f…Script SHA256: 8eb2a0e1…Results file: ch7_v4_main_v30_full57_results.jsonHardware: AMD Ryzen 9 9800X3D + RTX 4080 SUPER 16GB, Windows, C:_Phase2Run date: 2026-08-17 (execution) / 2026-08-18 (GT scoring & consensus) DOI lineage: 105th (NOT_ESTABLISHED) → 106th–107th (v12, observational) → 108th–109th (v20, exploratory) → 110th (v30 pre-registration) → 111th ★ (this document) Measurement is life. Description, not proof. Pre-registration protects us.PHILIA OS | 0∞1∞0.5∞
No takes yet. Share an insight, caveat, or question.
신두섭 (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: