Experimental evaluation demonstrates family-conditional performance of key-value cache eviction methods across seven model families, indicating the need to tailor scoring rules to architecture.
Key-value cache eviction is often evaluated as though one scoring rule should generalize across model families. We test that assumption under a prefill-time eviction contract in which the full prompt is available before selecting the retained cache positions. We introduce vorn, a residual-direction scoring signal that measures how strongly each position aligns with the direction toward which the model's residual computation is oriented, its residual facing, rather than relying directly on attention weights. Across seven model families spanning four laboratories (Mistral, Llama, Ministral, Gemma 2, Qwen 2.5, Gemma 4, Qwen 3-NT), eviction quality is family-conditional at a cache budget of 1,024 tokens. Residual-direction scoring attains or preserves near-ceiling retrieval on five families, while attention-weight scoring is stronger on two: Gemma 4 and Qwen 3 in the no-thinking configuration. Targeted comparisons using Ministral and Gemma 2 rule out attention topology and laboratory identity as sufficient explanations. We further evaluate token-level and sentence-level retention across multiple budgets. Sentence-level retention is frequently the dominant mechanism variable: it rescues attention-weight scoring on some families, only partially rescues it on others, and fails to rescue the disfavored residual-direction signal on Gemma 4 at constrained budgets. The results form a family- and budget-dependent rescue spectrum rather than a universal ranking of eviction methods. Within the non-floor, non-anomalous cells measured, sentence-level retention also reduces cost per correct answer. These findings support a bounded practical rule: choose the scoring channel according to model family and quality target, and treat retention granularity as a first-class design variable. Companion materials: Interactive companion: https://synapt.dev/vorn-mat/ Reproducible code + Docker pins: https://github.com/synapt-dev/vorn-mat HuggingFace dataset: https://huggingface.co/datasets/synapt/vorn-mat-cross-family-results
No takes yet. Share an insight, caveat, or question.
Layne Penney (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: