Break Later or Never: Prefix-Preserving Context Compaction for Coding Agents

Coding agents repeatedly invoke LLMs over growing, largely append-only histories (each request repeats the previous one and appends a new turn), making them natural beneficiaries of prefix caching. Yet the context compaction used to control this growth can undermine that benefit: rewriting earlier tokens invalidates reusable prefix KV and forces a costly re-prefill of the suffix (i.e., recomputing the KV of every token after the edit). We characterize this interaction across two models and two SWE-bench variants, finding that uncompacted agent trajectories achieve 98 to 99% prefix-cache reuse and that tool observations (e.g., file views and test logs) account for the largest share of context cost. As a simple prompt-level mitigation, batched observation masking (masking old observations once every few steps instead of every step) preserves 95 to 98% prefix reuse while reducing context length by up to 50%, serving cost by up to 18%, and task completion time by up to 21%. We then present ObsDrop, which moves compaction from the prompt into the KV cache: stale tool-observation KV is released while the logical token prefix remains unchanged, preserving prefix matching while reducing KV residency and attention work. We implement ObsDrop in SGLang and Sparse-vLLM and evaluate it across multiple attention backends and concurrent agent workloads. ObsDrop maintains task quality while improving cache residency, throughput, and tail latency under memory pressure. Its benefits grow with concurrency and KV pressure, reaching 2.45 to 2.85× higher throughput with FlashInfer (and up to 5.37× with the Triton backend under peak load), relative to no compaction. Our results show that effective context management for agents benefits from considering not only which tokens are retained but also how compaction interacts with the serving system.Code: https://github.com/musacim/ObsDrop