SAR2LiDAR: A deep learning-based cross-modal matching framework between SAR images and LiDAR point clouds

Synthetic aperture radar (SAR) and light detection and ranging (LiDAR) are two fundamental active remote sensing technologies. Their synergistic use can significantly enhance the effectiveness of remote sensing applications. However, existing studies primarily focus on product-level integration, and direct mapping between SAR images and LiDAR remains a challenge due to cross-dimensional modality differences. In this paper, a deep learning-based SAR-to-LiDAR (SAR2LiDAR) matching framework is proposed, which enables direct matching between SAR images and LiDAR point clouds. The framework features a dataset generation method that creates accurate 2D-3D correspondences between SAR images and LiDAR point clouds. It explicitly models the geometric and scattering visibility relationships of SAR imaging, and employs an autoencoder-based deep learning model with specific global feature representation and cross-modal alignment capabilities. The proposed model is trained and evaluated using airborne LiDAR datasets from five urban regions: Barcelona, Hong Kong, Los Angeles, Tarragona, and Girona. Extensive qualitative and quantitative experiments demonstrate that the proposed SAR2LiDAR framework achieves accurate and robust matching performance across diverse urban scenarios, attaining an average recall@1 of 95.64%, which significantly outperforms the best competing methods by 13%. Moreover, an average recall@1 of 91.68% on out-of-distribution (OOD) test regions demonstrates the strong generalization capability of the proposed framework and its potential for large-scale remote sensing applications.