What does this research mean for the field?

The frozen V1h10 evidence retrieval pipeline (TextAnalyzer CH7-V5) reproducibly achieves 86.0% strict consensus accuracy and demonstrates high robustness under live-evidence variation when evaluating LLM-generated claims. Novelty: ClaimNovelty.CONFIRMATORY. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

Assess the reproducibility and accuracy of the frozen CH7-V5 automated evidence retrieval pipeline on a 57-item corpus using live Wikipedia data.

August 30, 2026Open Access

PHILIA OS Phase 2 · 113th DOI Results Report: TextAnalyzer CH7-V5 — Evidence Retrieval with Dual-Pool Fetch, CPU Anchor, and Decidability Basis Labeling, 57-Item Full Corpus Confirmatory Study (Confirmatory, ESTABLISHED)

View Full Paper

Ask AI

Bookmark

Assess the reproducibility and accuracy of the frozen CH7-V5 automated evidence retrieval pipeline on a 57-item corpus using live Wikipedia data.
Evaluated the frozen V1h10 pipeline using fresh Wikipedia queries across a 57-item corpus categorized into L1 (N=20), L3 (N=25), and Hard (N=12) claims.
Assessed retrieval outputs against a 4-rater consensus protocol (Strict X) alongside automated detection metrics (FOUND+YES).
Compared item-by-item alignment across retrieval, basis, and match axes (Step4b comparison) under pre-registered confirmatory protocols.
Strict consensus criteria achieved 86.0% accuracy (49/57 items; 95% CI not reported), exactly matching exploration-phase benchmarks.
Step4b comparative analysis demonstrated 56/57 identical categorizations across found, basis, and match axes despite live-fetch variations.
Discrepancy was observed between automated metrics (FOUND+YES: 53/57, 93.0%) and strict 4-rater consensus (49/57, 86.0%), with zero premise-captured failures.

Implication

Confirmatory validation study shows 86.0% strict consensus accuracy across a 57-item test corpus, demonstrating pipeline robustness under live-evidence retrieval.

Abstract

DescriptionThis record contains the 113th DOI Results Report of the PHILIA OS research series (Trinity AI Research Team, Konyang University). CH7-V5 is a Wikipedia-based evidence retrieval pipeline that fetches factual information to support or refute LLM-generated claims, even when input claims are false. This confirmatory study applies the frozen V1h10 pipeline to the full 57-item corpus (L1×20 + L3×25 + Hard×12), A arm only, with fresh Wikipedia evidence. ★ Verdict: ESTABLISHEDC1 — A arm Strict X (4-rater consensus, Guide confirmed): 49/57 = 86.0% ≥ 49/57 (86.0%) ✅ N 8 items: L1_009 (FALSE STATED — degree adverb) · L3_007 (FALSE STATED — suffix drop) · L3_011 (FALSE STATED — sub-variant promotion, 2:2 → Guide N) · L3_014 (Selector failure) · L3_019 (FALSE STATED — modifier strip, 2:2 → Guide N) · L3_022 (CORRECTIVE) · L3_023 (CORRECTIVE) · H_005 (CORRECTIVE) Key findings: (1) Within-corpus live-fetch reproducibility confirmed: frozen V1h10 pipeline reproduced exploration-phase Strict X (49/57) exactly under fresh Wikipedia evidence. (2) Item-by-item Step4b comparison (56/57 identical across found/basis/match axes) demonstrates pipeline robustness under live-evidence variation. (3) No PREMISE_CAPTURED event observed. (4) L3_024 FALSE ABSENT: locus physically present; LLM comparator failed recognition. (5) Automatic metric gap: auto FOUND+YES = 53/57 vs Strict X = 49/57. Pre-registration analytic violations: 0. Documented procedural deviation: 1 (OUT_PATH/version-string separation; no pipeline-logic change). Confirmatory executable: ch7_V5_main_V1h10_57A_CONF.py | SHA256: 32433c6c… (LF)Corpus: philia_ch7_corpus_main_v3_patched.json | SHA256: 5fd61f8f…Pre-registration DOI: 10.5281/zenodo.22125648 (112th DOI)Hardware: AMD Ryzen 9 9800X3D + RTX 4080 SUPER 16GB, Windows, C:_Phase2 Measurement is life. Description, not proof. Pre-registration protects us.PHILIA OS | 0∞1∞0.5∞ | Trinity AI Research Team | Konyang University | 2026

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

신두섭 (2026) studied this question.

synapsesocial.com/papers/6a93f1736c1a8fb52e79e64d https://doi.org/10.5281/zenodo.22153519

View Full Paper

Ask AI

Bookmark

Implication

Confirmatory validation study shows 86.0% strict consensus accuracy across a 57-item test corpus, demonstrating pipeline robustness under live-evidence retrieval.

PHILIA OS Phase 2 · 113th DOI Results Report: TextAnalyzer CH7-V5 — Evidence Retrieval with Dual-Pool Fetch, CPU Anchor, and Decidability Basis Labeling, 57-Item Full Corpus Confirmatory Study (Confirmatory, ESTABLISHED)

Key Points

Implication

Abstract

Discussion

Cite This Study

PHILIA OS Phase 2 · 113th DOI Results Report: TextAnalyzer CH7-V5 — Evidence Retrieval with Dual-Pool Fetch, CPU Anchor, and Decidability Basis Labeling, 57-Item Full Corpus Confirmatory Study (Confirmatory, ESTABLISHED)

Key Points

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading