Calibration-free point cloud colorization for non-rigidly coupled LiDAR–camera systems

Non-rigidly coupled LiDAR-camera systems are increasingly adopted for 3D reconstruction and autonomous robots. However, point cloud colorization remains challenging for such systems due to the absence of fixed extrinsic parameters between sensors. This paper proposes a novel colorization framework for self-built non-rigid systems, which decomposes the colorization task into localized 3D-2D projective transformation estimation, mitigating the lack of fixed extrinsics. Each image then colors local 3D scene within its view frustum. When merging these scenes, overlapping regions yield multiple coloring candidates for individual 3D points. To resolve this, we introduce a coloring reliability assessment method that selects optimal coloring sources per point by evaluating projection geometry and reprojection errors. We conduct comprehensive experiments on self-collected outdoor real-world datasets, reporting the quantitative accuracy of 3D-2D transformation estimation and conducting qualitative analysis of colorized point clouds; we also perform ablation studies on the key module and cross-system validation on rigidly coupled systems, all of which collectively demonstrate the effectiveness and extensibility of the proposed method.