Exploring the Boundaries of Multi-Agent Ensembles: An Evaluation of Aggregation Operators (working paper, draft v8)

Working paper, draft v8 (12 October 2026), assembling the author's ten technical notes on language-model ensembles into one evaluation. For every aggregation operator (plurality voting, repeated sampling of one model, multi-round debate, synthesis, selection and certification by judges, routing by an orchestrator, decomposition into subtasks) the paper reports how much of the distance between the best single member and the oracle (the share of problems at least one member solves) the operator collects, and under which conditions. Data: a text corpus of 733 problems answered by seven models from five vendors with every jointly failed problem re-audited; the same corpus with the frontier models removed (a pre-registered debate, synthesis and judge run); 9,600 public debate traces; the SWE-bench Verified leaderboard matrix of 175 agent submissions, at the top and in mid-ranked accuracy bands; two public leaderboard matrices; released tau-bench and OS-Harm logs; and a cost-matched study of one model's draws. Findings: operators that only rearrange answers already produced collect nothing, whether the room above the best member is 2 points or 32, because the members fail on the same hard problems; a one-factor model of shared item difficulty reproduces their joint failures and their votes, and the independent Condorcet model over-predicts the vote by up to 27 points. Operators that add computation collect a few points under named conditions: repeated sampling where a single draw is below 0.9 at about ten times one answer's tokens; debate where the members' errors are unshared, at a loss of coverage; a judge panel whose sign against the best member depends on the tie rule. The one large prize nobody collects is the oracle on agent tasks, 10 to 32 points above the best submission, which a selector with an external verifier might reach. Not peer reviewed. Files: the paper (PDF and HTML, 15 sections, 20 tables, 87 references) and the landing page. Live page: https://alatip.github.io/paper-draft-aggregation-operators/