Investigating Energy Bounds of Analog Compute-in-Memory With Local Normalization

Modern edge AI workloads demand maximum energy efficiency, motivating the pursuit of analog Compute-in-Memory (CIM) architectures. Simultaneously, the popularity of large language models (LLMs) drives the adoption of low-bit floating-point formats that prioritize dynamic range. However, the conventional direct-accumulation CIM accommodates floating-point values by normalizing them to a shared, widened fixed-point scale. Consequently, hardware resolution is dictated by the input’s dynamic range rather than its precision, and energy consumption is dominated by the ADC. We address this limitation by introducing local normalization for each input, weight, and multiply-accumulate (MAC) output via aGain-Ranging MAC (GR-MAC). Normalization overhead is handled by low-power digital logic, enabling the computationally expensive MAC operation to remain in the energy-efficient low-precision analog regime. Energy modelling shows that adding a gain-ranging stage to the MAC increases the input dynamic range by 4 bits without increasing energy consumption at a 35 dB SQNR standard. Additionally, the ADC resolution requirement becomes invariant to input distribution assumptions, allowing the construction of an upper bound with a 1.5-bit reduction compared to the conventional lower bound. These results establish a pathway to unlocking favourable energy scaling trends in analog CIM for modern AI workloads.