Automatic calibration and association for roadside radar and camera based on fluctuating traffic volume

Abstract Accurate perception of the movement and appearance of vehicles depends on the robustness and reliability of the extrinsic parameters calibration in a multi-sensor fusion scenario. However, conventional calibration methods require manual acquisition of prior information, leading to high labor costs and low calibration accuracy. Therefore, we proposed an automatic coarse-to-fine calibration method for roadside radar and camera sensors to lower costs and improve accuracy. Next, an association strategy based on fluctuating traffic volumes was also developed to assist in robust target matching during the coarse-to-fine calibration process. Finally, extrinsic parameters between the radar coordinate system and camera coordinate system were calibrated through double rotations of the position vectors obtained from each system. To verify the proposed method, an experiment was conducted on a pedestrian bridge using an uncalibrated 4D millimeter-wave radar and a traffic monocular camera. The results showed that our proposed method reduced the interquartile range of the roll angle by 41.5% compared to a state-of-the-art neural network method. It also outperformed the manual calibration method by 2.47% in terms of the average reprojection error.