Answers are grounded in this paper and cite their sources.
Agentic enterprise benchmarks increasingly use executable tasks, hidden reference states, and automated judges to determine whether an AI system has completed consequential work. Yet an internally consistent benchmark can produce a false, unsupported, or institutionally inadmissible completion label when its task predicate, observation route, case identity, source lineage, authority, or target-world mapping is defective. This paper proposes the Oracle Independence Trial (OIT), a bounded, claim-relative protocol for auditing the warrant carried by a benchmark oracle rather than treating its reference label as unchallengeable truth. Building on outcome verification, evidence continuity, and adversarial cross-Field completion research, OIT distinguishes reference-world agreement, construct adequacy, source independence, real-world effect observation, and legitimate release authority. We specify typed support records, conditional propositions, counterexamples, and a prospective information-matched evaluation against strong conventional audit engineering. A deterministic reference demonstration generates 240 author-constructed cases across twelve failure families. A second, independently coded local audit reproduces its outputs, compares SQLite and Python adjudicators, checks 384 reduced states and 4,000 seeded perturbations, and exposes two limits: unbound cross-case witnesses can invert completion verdicts, while apparently well-addressed but false witness content can deceive both conventional and typed policies. An original assertion-based error-handling defect is disclosed. No live-agent experiment, independently verified oracle, measured Field-specific advantage, or production validity is claimed. The resulting contribution is a falsifiable framework for testing what justifies the authority of ground truth in enterprise-agent evaluations—and when that authority must remain explicitly unresolved.
No takes yet. Share an insight, caveat, or question.
Rogério Figurelli (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: