EVOKE: A KV-Cache Memory Hierarchy with Recompute-Free Block Recovery for LLM Serving

Production LLM serving treats cache overflow as a dead end: under budget pressure, truncate the oldest history or re-prefill on every call. EVOKE (EVict and recOver KV cache Entries) makes eviction reversible. A recompute-free splice writes the saved K and V tensors back into the active cache with a single RoPE rotation; the cost is the tensor transfer. Similarity scores select which evicted block to recover; the splice then returns that block’s bytes at a new logical position, byte-identical to what the model originally attended. Recovery is keyed by block identity (where RAG would substitute re-encoded text). We add three KV-paging primitives to a llama.cpp fork (block save, block load with RoPE re-anchoring, per-layer attention capture), wrapped by a Python policy and evaluated across four model families on a single 16 GB consumer GPU. The save+load round-trip runs 5.9 to 7.5×faster than re-prefilling the same token range on the same engine (Qwen 2.5 7B), 2.6 to 2.8×on Llama 3.1 8B; load-only is 20 to 32×and 7 to 15×respectively. Across three benchmarks (NIAH, multi-fact, agentic), every recovery-less baseline stays at or below 43% multi-fact pass rate while every recovery-bearing policy reaches at least 48%, holding at b=1024 on hybrid Mamba/Attention Qwen 3.5 9B (68%) and MoE Qwen 3.6 35B-A3B (52%). Recency, StreamingLLM, H2O (Zhang et al., 2023), and SnapKV (Li et al., 2024) are same-harness reimplementations of their published configurations. Within the recovery-bearing cluster, a 15-seed sweep against InfLLM (Xiao et al., 2024b) (K=8, same splice) shows a budget-axis crossover: InfLLM wins at b=512 on both architectures (non-overlapping Wilson CIs), the two tie at b=1024, and EVOKE’s point estimates lead at b=2048 (CIs overlap, Section 7.3). The splice is the load-bearing contribution; scheduler choice and its Kare regime-dependent parameters.