Controlled evaluation reveals divergent token savings across compression levers in multi-turn coding agents, indicating single-shot compression benchmarks do not reflect end-to-end costs.
Context compression is widely proposed as a way to cut the token bill of LLM-based coding agents, and public benchmarks report that aggressive compression preserves task-solving quality. Yet these two facts do not imply the third one everybody assumes: that compressing file reads actually saves money in a real multi-turn agent. We instrument a production compression gateway (Paritok) sitting between coding agents (Claude Code, Codex) and frontier LLMs (Claude Sonnet, GPT-5), and decompose the token bill of real agent sessions into three independent levers: (i) tool-schema filtering, (ii) content compression of file reads and tool output, and (iii) history summarization. Measuring each lever in isolation across controlled A/B runs, we find that the three levers save money at fundamentally different rates. Tool-schema filtering saves a fixed block of tokens every turn (linear in turn count N) and is the only lever that is unambiguously and reproducibly positive; on a typical turn it removes 21K-57K tokens. Content compression, by contrast, saves only ~2% of the (cache-priced) prefix per turn - but because compressed reads accumulate in history and are re-sent on every later turn, its cumulative saving grows quadratically, approximately 3,350 N^2 tokens (measured), overtaking the fixed tool-filter saving within the first ~6 turns before the context window caps it. Crucially, a non-destructive gateway lets the agent pull the original bytes back on demand, and whenever it does the recall simply re-sends the bytes just compressed away, so each recall costs a fixed, bounded amount - the length of the one compressed segment it re-sends, never a multiplicative blowup. The more the agent recalls, the more of the accumulated content saving is spent back, one compressed segment at a time. Finally, we show that a strong single-shot compression benchmark - the 86.5% SWE-bench quality retained at a 25.7% compression rate achieved by the compression model the gateway deploys (Paritok-4B, reported separately) - is orthogonal to multi-turn agent cost and must not be cited as a cost-saving argument for the gateway. We distill these into an actionable recipe for where token-saving effort actually pays off.
No takes yet. Share an insight, caveat, or question.
Chen et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: