AdapTQ-LA: Layer-Aware Asymmetric KV-Cache Quantization via Fast Walsh-Hadamard Preprocessing

AdapTQ-LA is a streaming KV-cache quantization approach for large language model inference that combines randomized Fast Walsh-Hadamard Transform (FWHT) preprocessing with Lloyd-Max scalar quantization. The proposed Layer-Aware Asymmetric (AdapTQ-LA) configuration protects the first transformer layer using FP32 storage and assigns higher precision to Keys than Values. The evaluated implementation uses C++17 with AVX2 support and integrates with Hugging Face Transformers through a custom DynamicCache. On Qwen2-0.5B at a context length of 512, the evaluated K4-V2 configuration with Layer-0 FP32 achieves 1.896 MB of KV-cache memory compared with 7.500 MB for the dense FP16 baseline, corresponding to approximately 3.95× memory reduction, with perplexity of 1.049 compared with 1.007 for the dense baseline. The work also reports ablations of the Rademacher/FWHT preprocessing, asymmetric K/V precision allocation, Layer-0 sensitivity, clipping, and other experimental directions. End-to-end generation speedup is not demonstrated in the current Hugging Face integration, and the study does not provide a direct matched comparison with TurboQuant.