Modeling and optimization of energy-efficient CNN hardware accelerators with mixed-precision quantization

Efficient design and optimization of configurable mixed-precision hardware accelerators of convolutional neural networks (CNN) require (1) a fast and flexible software tool to determine hardware properties of a given accelerator configuration and (2) an efficient optimization algorithm to search in the challenging space of configurations. We propose a framework capable of modeling a broad class of CNN inference accelerators and determining the configurations leading to the most suitable trade-offs among the accuracy, energy, latency, and memory size. The framework extends the state-of-the-art tool (Timeloop) to support mixed-precision quantization, precision-configurable arithmetic operations, approximate arithmetic operations, bit packing, and energy estimation. The extended Timeloop connected with a multi-objective genetic algorithm allowed us to reach energy-efficient configurations of selected inference accelerators (Eyeriss v1 and Simba) evaluated on hundreds of CNNs (MobileNet V1 and V2, ResNet-50, and automatically generated CNNs), classifying CIFAR-10 and ImageNet data sets. It was also revealed that supporting only two levels of bit precisions in arithmetic units is sufficient for obtaining the best-scored CNN implementations.