A long-running agent accumulates context (files read, tool output, search results) faster than a fixed KV budget can hold, and both standard responses lose something: truncation discards content the agent may need again, while re-prefill re-encodes it at full forward-pass cost every time the agent refers back. EVOKE (EVict and recOver KV cache Entries) makes eviction reversible. An evicted block’s K and V tensors move to host RAM; when the agent needs the block again, a recompute-free splice writes the saved tensors back into the live cache with a single RoPE re-anchoring rotation. Agent context arrives as identifiable blocks (a file read, a tool result), so recovery is keyed by block identity: the spliced bytes are the tensors the model first computed, never re-encoded text. We add the required KV-paging primitives to a llama.cpp fork, build an eviction and recovery policy on them, and evaluate four model families on a single 16 GB consumer GPU. A recovered block is as usable for recall as a fresh re-decode (100 % vs 100 %; the evicted-not-restored floor is 0–5 %), at 5.9–7.5× lower lifecycle cost than re-prefill on Qwen 2.5 7B. Across three recall benchmarks, every recovery- less baseline (recency, StreamingLLM, H2O (Zhang et al., 2023), SnapKV (Li et al., 2024)) fails once the probed content has been evicted, while every recovery- bearing policy passes; the divide repeats at the one budget swept (b=1024) on hybrid Mamba/Attention and MoE-with-thinking models. Against a same-substrate InfLLM (Xiao et al., 2024b) adaptation the two schedulers trade places along the budget axis: InfLLM separates at b=512, the two tie at b=1024, and EVOKE leads at b=2048 with overlapping intervals. This revision also adds a forward- looking eviction signal outside the attention-history family: a ridge probe distilled from the Jacobian-lens workspace readout of Gurnee et al. (2026) scores blocks by content at prefill time, ranks the planted fact above SnapKV’s and H2O’s choices offline on all three models it was run on (fact AUC 0.891 vs. 0.622 on Qwen 2.5 7B, replicating at 0.865 and 0.939 on Qwen3-8B and Qwen3-4B), and keeps it resident at every swept budget with no recovery path and no measurable decode overhead. We also measure where a recovered block should land: tail re-anchoring helps only near a long-context model’s far edge, is neutral at normal distances, and hurts short-context models. Recompute-free recall of evicted KV follows ArkVale (Chen et al., 2024); EVOKE contributes the agent-session regime, identity-keyed addressing, primitives spanning pure-attention, hybrid, and MoE architectures, and the placement measurement.
Anish Shrestha (Thu,) studied this question.