Energy-Efficient Neural Network Inference through Golomb-Rice Compression of Activations for Edge Devices

This is the authors' abstract. We don't add key points for this paper.

A key challenge for Deep Neural Network (DNN) inference on resource-constrained edge devices is the high energy consumption caused by frequent memory accesses for parameters. While our previous research has demonstrated the efficacy of data compression for weights, this paper extends our approach to include on-the-fly compression and decompression of activations. We propose a comprehensive hardware solution comprising two main components: a Golomb-Rice (GR) compression system and an Output Activation Processing Module (OAPM). The GR system provides an efficient activation compression mechanism, while the OAPM enables dynamic data-format capabilities for handling activations. Additionally, we present an enhanced Input Activation Extract Module (IAEM) with an integrated decompression unit and dynamic activation processing capabilities. When integrated with an industry-strength Neural Network accelerator and evaluated using the Anomaly Detection (AD) TinyML benchmark, our lossless compression system achieved a 2.3× compression ratio, reduced memory bandwidth usage by 49.28%, and improved inference speed by 10%.