An efficient framework for creating depth annotated image datasets with camera and Lidar fusion

We identified two challenges in the current creation of depth-annotated dataset for autonomous driving or 3D reconstruction: 1) The viewing angle difference between cameras and radar, causes projecting background points onto foreground objects in the Lidar-to-image projection process, leading to inaccurate depth information acquisition. 2) The substantial manpower required to eliminate occluded points during the depth dataset creation process. In addressing these challenges, we analyzed the principles behind occlusion and proposed an automated filtering method for point cloud projection images. Our method involves segmenting images into various-sized sliding windows based on different scenes to filter out occluded points. Through this approach, we achieve effective occlusion removal in diverse scenarios, encompassing structured objects like cars, buildings, pedestrians, as well as unstructured objects like trees. Moreover, the proposed algorithm automates the process, thereby significantly reducing both labor and time costs. Additionally, we proposed different schemes for creating sparse and dense depth maps based on the density of the camera depth image. We have open-sourced the code, and the repository can be found at: squirreljj/An-efficient-framework-for-creating-depthannotated-image-datasets-with-camera-and-Lidar-fusion- (github.com)