Cross-Model Agreement as an Escalation Signal for LLM Cascades

Cascaded LLM systems answer with a cheap model and escalate hard queries to an expensive one. We test whether agreement between two cheap models from different families is a reliable, training-free signal for escalation, and when acting on it saves compute. Llama-3.1-8B and Qwen3-8B answered 1,100 numeric questions from GSM8K, MATH-500 and FinQA; questions on which they disagreed went to Qwen3-32B. All models ran on the same two NVIDIA T4 GPUs and cost was measured in GPU-seconds. Agreement is a strong signal: Llama was right on 93.3% of questions when the pair agreed and 13.1% when it disagreed (AUROC 0.90). Escalating the 34% of disagreements captured 88% of the gain an oracle could achieve over random escalation, and the router matched the strong model's accuracy (86.3% vs 85.6%). Whether it saves compute depends on accounting: 17% less than always using the strong model if the second cheap model is already running, 8% more if it must be paid for. A pre-registered break-even rule wrongly predicted savings, because escalated questions are the expensive ones: the strong model wrote 2.0× as many tokens on them. We derive a corrected rule, show that routing pays once the strong model costs at least 3.7× the cheap anchor, and give a price list that tells practitioners which disagreements to escalate first. We also show that a single prompt instruction can fake or hide a routing result.