Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving

Detecting objects such as cars and pedestrians in 3D plays an indispensable in autonomous driving. Existing approaches largely rely on expensive LiDAR for accurate depth information. While recently pseudo-LiDAR has been as a promising alternative, at a much lower cost based solely on images, there is still a notable performance gap. In this paper we substantial advances to the pseudo-LiDAR framework through improvements stereo depth estimation. Concretely, we adapt the stereo network and loss function to be more aligned with accurate depth of faraway objects --- currently the primary weakness of-LiDAR. Further, we explore the idea to leverage cheaper but extremely LiDAR sensors, which alone provide insufficient information for 3D, to de-bias our depth estimation. We propose a depth-propagation, guided by the initial depth estimates, to diffuse these few exact across the entire depth map. We show on the KITTI object detection that our combined approach yields substantial improvements in depth and stereo-based 3D object detection --- outperforming the previous-of-the-art detection accuracy for faraway objects by 40%. Our code is at https: //github. com/mileyan/PseudoLidarV2.