A method for matching low-resolution radar point cloud with optical image information

The fusion of millimeter-wave radar and optical camera is a challenge in the multi-sensor data fusion field. In this paper a new space calibration algorithm is proposed to match the low-resolution radar point cloud information and the optical image information. This paper proposes that fusing the wheeled odometer and radar to increase the density of point cloud. Moreover, the real 3D model information of marker and radar point cloud are fitted to extract the 3D feature point of marker, and the 2D pixel feature point of the marker is extracted by Instance Segmentation algorithm. Then, the marker's 3D-2D matching points are used to solve the external parameters between the radar and camera. Based on Turtlebot2 hardware platform, the experimental results show that the proposed method can calibrate the radar and camera, and match the radar point cloud and optical image information under the condition of large resolution difference. The proposed algorithm has practical application value in autonomous driving, such as target detection and tracking, scene 3D reconstruction, path planning and navigation.