NICE: Deep Neural Network Acceleration via Hardware-Friendly Index Assisted Compression

The exponential scaling of Large Language Models (LLMs) has exposed a growing mismatch between computational demands and hardware efficiency. Although model compression is essential for mitigating this gap, two bottlenecks fundamentally limit its effectiveness in practice: (1) high-magnitude activation outliers that degrade the accuracy of conventional uniform quantization, and (2) dynamically varying activation sparsity that is difficult to exploit with rigid hardware datapaths. Existing index-based schemes, such as mixed-precision or non-uniform quantization, often incur prohibitive energy and area overheads due to the complex decoding logic required at runtime. In this paper, we propose NICE, an index-assisted algorithm-hardware co-design framework that achieves high-efficiency DNN acceleration by synergistically exploiting unstructured weight sparsity and centroid-based activation indexing. The core philosophy of NICE is to reformulate computation-intensive Multiply-Accumulate (MAC) operations into a unified table-lookup (LUT) mechanism. Specifically, we design a new hardware primitive that treats static weight sparsity and dynamic activation indices as lookup coordinates, enabling the direct retrieval of pre-computed or quantized results. Building on this, we develop the NICE Architecture, a systolic array-based accelerator that introduces novel dimensions of data-and-index reuse, effectively alleviating the data-movement bottlenecks of traditional architectures. Experiments show that NICE can achieve 51.3% energy reduction and 3.83x speedup on average.