SGM3D: Stereo Guided Monocular 3D Object Detection

Monocular 3D object detection aims to predict the object location, dimension and orientation in 3D space alongside the object category given only a monocular image. It poses a great challenge due to its ill-posed property, which is a critical lack of depth information in the 2D image plane. While existing approaches leverage off-the-shelf depth estimation or rely on LiDAR sensors to mitigate this problem, dependence on the additional depth model or expensive equipment severely limits their scalability to generic 3D perception. In this paper, we propose a stereo-guided monocular 3D object detection framework, dubbed SGM3D, which aligns the monocular feature to the stereo feature and exploits the network's ability of generating the cross-modal feature for accurate 3D object detection. We present a multi-granularity feature alignment (MG-FA) mechanism to exploit the network's ability to generate stereo-mimicking features given only monocular cues. Coarse feature-level, as well as fine anchor-level feature alignment, are both leveraged for monocular feature guidance. In addition, we introduce an IoU matching-based feature alignment (IoU-MA) method for object-level feature alignment between the stereo and monocular predictions to alleviate the mismatches while adopting the MG-FA. Extensive experiments on the KITTI datasets demonstrate the effectiveness of our method. Code is publicly available at https://github.com/zhouzheyuan/sgm3d .