🚀 TTAS-X2: Extreme KV Cache Compression for Large Language Models

We introduce TTAS-X2, a novel KV cache compression method that treats Keys (K) and Values (V) with fundamentally different strategies based on their functional roles in attention. While previous approaches compress K and V uniformly, we show that: K is direction‑sensitive and must be preserved with high fidelity V tolerates aggressive compression without harming attention scores By applying 4‑bit scalar quantization to K and hierarchical product quantization (256 → 64) with Hadamard transform and fixed‑budget outliers to V, we achieve: Attention cosine similarity ≥ 0.8 across all layers of Qwen3-32B Compression ratio > 10× for KV cache BPW < 1.6 while maintaining functional behavior This work enables running 32B‑class models on devices with 8GB RAM, previously impossible. 🧠 1. Introduction KV cache is the main memory bottleneck during long‑context inference.Standard quantization (2‑3 bits) degrades attention scores severely because K is distorted, and softmax is highly sensitive to directional changes. We propose a functional separation: K: 4‑bit scalar + per‑head scale V: RPQ + Hadamard + fixed‑budget outliers ⚙️ 2. Methodology 2.1 K Compression For each head, we store a 4‑bit quantized version with a per‑head scale.No Hadamard, no PQ, no residuals – direction is preserved. 2.2 V Compression RMS normalize per vector Hadamard transform to spread energy Two‑level product quantization (K1=256, K2=64–512) Fixed‑budget outliers (32–256 per head) 📊 3. Experiments Model: Qwen3-32B (head_dim = 128, 64 layers, GQA with 8 KV heads)Sequence length: 1024 tokens per test Layer Cosine OUTLIER_BUDGET K2 0 0.824 32 64 1 0.804 64 128 2 0.803 128 256 4 0.806 256 512 All layers achieve cosine ≥ 0.8, proving that TTAS-X2 generalizes across depth. 💾 4. Memory Analysis For a 32B model with 64 layers and GQA (8 KV heads): Context Length Original (FP16) TTAS-X2 Ratio BPW 4k 1.07 GB 0.17 GB 6.3× 1.59 16k 4.29 GB 0.68 GB 6.3× 1.59 32k 8.59 GB 1.36 GB 6.3× 1.59 64k 17.18 GB 2.72 GB 6.3× 1.59 TTAS-X2 enables 32k context on 8GB devices for the first time. 🔬 5. Conclusion We demonstrate that functional separation of K and V is the key to extreme KV cache compression.TTAS-X2 preserves attention fidelity while reducing memory footprint by an order of magnitude. Patent pending – all rights reserved. 📎 6. References Multiverse Computing, "CompactifAI" (2025) Intel, "KV Cache Compression via Token Pruning" (2024) Original TTAS work (Alsheck, 2025) 📬 Contact Abdulaziz Alsheck📧 abdulazizabdullahalalsheck@gmail.com Phone Number +966560756695