Optimized Convolutional Neural Network at the IoT edge for image detection using pruning and quantization

Abstract Most real-time computer vision applications heavily rely on Convolutional Neural Network (CNN) based models, for image classification and recognition. Due to the computationally and memory-intensive nature of the CNN model, it’s challenging to deploy on resource-constrained Internet of Things (IoT) devices to enable Edge intelligence for real-time decision-making. Edge intelligence requires minimum inference latency, memory footprint, and energy-efficient model. This work aims to develop an energy-efficient deep learning accelerator using a 3-stage pipeline: Training, Weight-pruning, and Quantization to reduce the model size and optimize the resources. First, we employ YOLOv3, a CNN architecture to detect objects in an image on the trained data. In addition, a sparse network of YOLO has been created by using pruning, which helps to improve the network’s performance and efficiency by reducing the computational requirements. Finally, we utilize 8-bit quantization to reduce the precision of the weights and activations, in a neural network. The evaluation of our proposed model shows that combining pruning and 8-bit quantization improves the efficiency and performance of the model. While pruning shows a decline of 80.39% in model parameters. The combination of 8-bit quantization results in an improvement in inference latency by 22.72% compared to existing SQuantization approach and a reduction of energy consumption by 29.41%.