A lightweight optimization framework for real-time object detector on the embedded GPU platform

Abstract At present, embedded GPU devices play a central role in carrying edge intelligence applications, especially object detection. However, efficiently deploying and inferring convolutional neural network (CNN)-based object detection models on embedded edge GPU devices with small size, low power consumption, and limited computing resources remains a significant challenge. In this paper, we proposed a lightweight optimization framework for YOLOv3 to improve its inference efficiency onto a small GPU-based edge device, in this case the Jetson Nano, which is a low-cost entry-level artificial intelligence (AI) computer for the embedded and edge intelligence markets. First, we replace the Backbone of YOLOv3 by Efficient-RepVGG, which is more suitable for GPU architecture. Secondly, a lightweight and efficient bidirectional feature pyramid network is designed to further compress the model. Finally, the Optimized Couple Head is proposed to compensate for the loss of accuracy without affecting the effect of model lightweight. After the improvement, the model saves 92.26% and 92.10% of the parameters and computation, increasing the inference speed by 9.95 times, while maintaining high object detection accuracy. Compared with other lightweight object detectors, the inference speed of Optimized YOLOv3 is about 1.6 to 2 times faster while maintaining similar detection accuracy; and the detection accuracy is 3 to 6 percentage higher while maintaining similar inference speed.