Uncertainty-Aware Projection Network for Multi-View 3D Object Detection

In recent years, multi-view 3D object detection has drawn a lot of interest in autonomous driving. The projection from 2D image point to 3D world space is the core step of multi-view 3D object detection. However, the uncertainty of depth estimation and camera parameters can make the projected 3D position inaccurate, leading to a performance drop. To this end, we propose an Uncertainty-aware Projection Network (UPN), including a depth uncertainty-guided 3D point generation module and a camera-aware 3D position encoder. For the depth uncertainty-guided 3D point generation module, we predict the depth as a distribution to generate depth uncertainty-aware 3D points. For the camera-aware 3D position encoder, we alleviate the uncertainty of camera parameters by employing adaptive receptive fields to refine the position embedding. Based on these two modules, the proposed UPN can effectively deal with the performance drop caused by inaccurate 2D-3D projection. To the best of our knowledge, this is the first work to model the 2D-3D projection uncertainty for multi-view 3D object detection. The proposed UPN achieves 56.9% mAP and 65.4 % NDS on the nuScenes test dataset, demonstrating the superiority of our method.