What question did this study set out to answer?

The aim is to assess the effectiveness of key-value cache eviction methods across different model families and establish the need for family-specific approaches.

June 5, 2026Open Access

Vorn: Residual Direction, Familial Eviction, and the Granularity Rescue Spectrum

Key Points

The aim is to assess the effectiveness of key-value cache eviction methods across different model families and establish the need for family-specific approaches.
Analyzed eviction performance across seven model families including Mistral, Llama, Gemma, and Qwen.
Conducted targeted multi-budget falsification probes to rule out attention-pattern influences and laboratory identity differences.
Developed a complementary granularity rescue spectrum to report the unit-of-retention effect.
Five model families demonstrated effective long-context retrieval under residual-direction-based eviction (vorn).
Two families, Gemma 4 and Qwen 3-NT, showed reduced retrieval competence, favoring attention-based mechanisms.
Eviction-method effectiveness depends on family quality and retention granularity rather than just scoring criteria.

Abstract

Key-value cache eviction is a major approach to long-context inference, and prior work typically selects a scoring criterion under the implicit assumption that its ranking generalizes across model families. Under a prefill-time cache selection contract with full-prompt visibility, we show this assumption fails: KV-cache eviction effectiveness is family-conditional. Across seven model families spanning four laboratories (Mistral, Llama, Ministral, Gemma 2, Qwen 2.5, Gemma 4, Qwen 3-NT), five families attain or preserve long-context retrieval competence under residual-direction-based eviction, which we call vorn, while two families (Gemma 4 and Qwen 3-NT) are attention-favoring at the shared b=1024 gate. Two targeted multi-budget falsification probes rule out attention-pattern and laboratory identity as sufficient explanations. A complementary granularity rescue spectrum reports a unit-of-retention effect across family-anchored sentence-attention surfaces: retrieval competence under eviction ranges from rescue-resilient (Mistral) through threshold-bounded recovery (Llama 3.1) to rescue-resistant (Gemma 4). The practical implication is that eviction-method selection should be family-conditional on the quality axis, while cost-per-correct is dominated by retention granularity rather than scoring channel. Companion materials: Interactive companion: https://synapt.dev/vorn-mat/ Reproducible code + Docker pins: https://github.com/synapt-dev/vorn-mat HuggingFace dataset: https://huggingface.co/datasets/synapt/vorn-mat-cross-family-results

Read Full Paperexternally

AI에게 질문

Bookmark

View Full Paper