Answers are grounded in this paper and cite their sources.
Assesses judgment quality in production work using peer aggregation and debate, indicating improved outcomes.
Standalone LLM judgment on production work is uneven across model families. We report a focused multi-model judgment benchmark on six experts from different providers: standalone; aggregation (peer outputs as context); and structured adversarial debate (forced for-and-against, then verdicts). Judgment quality means production weighted reference-finding coverage (leave-one-out consensus alignment; not precision) and GPQA Diamond accuracy. We ask whether peer context and debate are associated with higher observed scores than standalone. A companion process paper covers fail-closed gates. Production false-positive burden is unmeasured (Limitations). On GPQA Diamond (known answers), the panel moves from 77% standalone to 86% with aggregation. Under mixed expert-plus-light post-debate context, panel mean accuracy is 88% after debate—not a pure expert continuation of77–86—so under that non-matched condition debate adds little beyond aggregation. On production, each expert’s aggregation coverage is 3.7 to 12.5 percentage points above standalone. Every expert’s aggregation point estimate is higher: the observed floor, mean, and top all move up. Post-debate scores are 26.1 to 49.2 percentage points above the same standalone baseline (42 50% to 68–94%; panel +39.9pp)—recognition coverage under a leave-oneout consensus baseline, not independent ground truth (Limitations). Thus production is where the large observed coverage difference appears, with that caveat. Scorer and finding-extractor roles overlap with panel contestants. Associated with higher observed scores; no ablated causality over extra tokens or majority selection alone.
No takes yet. Share an insight, caveat, or question.
Igor Razumnyy (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: