LKC-CXL-PIM: Long KV Cache via CXL Processing-in-Memory

Large Language Models (LLMs) have demonstrated unprecedented capabilities in natural language understanding and generation tasks. However, extending their context window for long-sequence tasks severely exposes the memory wall problem, driven almost entirely by the massive size and memory bandwidth requirements of the Key-Value (KV) cache. While traditional Processing-in-Memory (PIM) solutions attempt to alleviate bandwidth bottlenecks by pushing computation close to data, their reliance on complex Floating-Point (FP16) compute logic and frequent dequantization limits their efficiency. In this paper, we propose LKC-CXL-PIM (Long KV Cache via Compute Express Link Processing-in-Memory), a hardware-software co-designed architecture that achieves memory expansion and near-memory attention computation with unprecedented efficiency. Our core contributions include a highly efficient Integer Non-Linear Unit (iNLU) and an Outlier-Aware Logic data path, which perform the attention Softmax using exclusively integer operations for the vast majority of activations, seamlessly handling the $1%$ outliers through an overflow buffer. Evaluated on a highly customized, cycle-accurate Ramulator 2.0 platform, LKC-CXL-PIM significantly outperforms state-of-the-art baselines. Across context lengths from 2K to 128K, our architecture reduces total read access cycles by ~98.9% (e.g., from 10.8M to 0.12M cycles for 8K context), reduces write queuing delays similarly, and virtually eliminates row buffer misses compared to conventional memory hierarchies. LKC-CXL-PIM demonstrates a highly scalable, energy-efficient path forward for deploying long-context LLMs.