A 28-nm 3.32-nJ/Frame Compute-in-Memory CNN Processor With Layer Fusion for Always-on Applications

This work presents an always-on CNN processor featuring compute-in-memory (CIM) and layer-fusion (LF) techniques. It demonstrates an end-to-end neural network (NN) inference while eliminating memory accesses for both the weight and inter-layer activation, thus significantly saving associated energy. The analog LF-CIM processor combines charge- and time-domain multiply-accumulate (MAC) arrays in a fusional manner, avoiding the need for the interfacial analog-to-digital and digital-to-analog converters; it also supports a highly sparsity-adaptive network and exhibits low-voltage-supplied tolerance to circuit noise and variations, which facilitates an energy-efficient and robust CNN processor, simultaneously. Furthermore, the full-precision activation originating from the LF reduces the NN parameters and operations, while the inference accuracy is improved by the presented ensemble NN. The prototype processor is fabricated in a 28-nm CMOS process, demonstrating a fully on-chip MNIST inference with 97.9% accuracy. It operates at 3,508 frames per second while consuming <tex-math notation="LaTeX">11.6~μ </tex-math>W at a 0.5-V supply; the achieved efficiency of 3.32 nJ/frame is over 50-fold than the state-of-the-art end-to-end MNIST accelerators.