End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds

Recent work on 3D object detection advocates point cloud voxelization in-eye view, where objects preserve their physical dimensions and are separable. When represented in this view, however, point clouds are and have highly variable point density, which may cause detectors in detecting distant or small objects (pedestrians, traffic signs,. ). On the other hand, perspective view provides dense observations, which allow more favorable feature encoding for such cases. In this paper, we to synergize the birds-eye view and the perspective view and propose a end-to-end multi-view fusion (MVF) algorithm, which can effectively learn utilize the complementary information from both. Specifically, we introduce voxelization, which has four merits compared to existing voxelization, i) removing the need of pre-allocating a tensor with fixed size; ii) the information loss due to stochastic point/voxel dropout; iii) deterministic voxel embeddings and more stable detection outcomes; iv) the bi-directional relationship between points and voxels, which lays a natural foundation for cross-view feature fusion. By dynamic voxelization, the proposed feature fusion architecture each point to learn to fuse context information from different views. operates on points and can be naturally extended to other approaches using point clouds. We evaluate our MVF model extensively on the newly released Open Dataset and on the KITTI dataset and demonstrate that it improves detection accuracy over the comparable single-view baseline.