Ultra High-Definition Video AI Inference For 8K Object Detection on Edge Devices

Video artificial intelligence (AI) inference, especially object detection from 8K ultra high-definition video, on edge devices is becoming increasingly necessary due to the miniaturisation of 8K cameras and the growing focus on security and privacy protection. To detect objects in real-time in 8K video, both large and small objects need to be detected with limited computational resources on edge devices because the size of objects in distant and near views differs greatly within the same video frame. In the conventional methods for high-definition videos, an object detection model is applied to the regions where objects are expected to exist using density estimation or cluster detection models. The conventional object detection methods, however, have two issues. The first is that the object sizes are not considered when cropping regions from images. The second is that, depending on the scene, the number of cropped regions can be large, making them difficult to process in real time on edge devices. To address these issues, we propose a novel object detection method that can be performed in real-time on edge devices using a density estimation model trained with size-aware ground truths and a technique that reduces computational complexity per frame while maintaining the detection accuracy. We evaluated our proposed method with an 8K video dataset and the Jetson AGX Orin, a typical edge device. In the evaluation with an 8K video dataset, our method achieved of mean average precision (mAP) 46.9%, 1.68 times higher than that of the conventional method (28.0%), while requiring comparable computational complexity for object detection. Our method also achieved a processing speed of nearly 4 fps, which is required for various applications on edge devices, with only about 2% lower accuracy than when rectangle regions were not selected on the Jetson AGX Orin.