Extrinsic auto-calibration of LiDAR-camera system based on deep learning

Precise geometrical extrinsic calibration is a prerequisite for fusing LiDAR and Camera information. Most existing calibration techniques necessitate a significant amount of calibration target data and human effort, resulting in a tedious and laborious procedure. Moreover, the relative poses between the LiDAR and the camera may drift cumulatively as the application time increases, and even the extrinsic parameters can change drastically following an accidental impact. This paper builds an extrinsic calibration end-to-end differentiable iterative refinement network. We combine the neural network with the geometric constraints of extrinsic calibration. Our model is trained entirely on the synthetic dataset and performs well when directly applied to the real-world dataset without fine-tuning, demonstrating our method's strong generalization capability.