Uni-Map: unified camera-LiDAR perception for robust HD map construction

High-definition (HD) map construction is essential for autonomous driving in providing precise and comprehensive static environmental information, typically relying on camera, LiDAR, or fusion-based configurations. While fusion methods offer superior performance, current approaches require separate models for each sensor setup, leading to high deployment costs. However, current methods face two major issues: high costs due to separate training and deployment for each input configuration, and low robustness when sensors are missing or corrupted. To address these challenges, we introduce Uni-Map, a Unified Robust HD Map Construction Network, a single model designed to perform well across camera-only, LiDAR-only, and camera-LiDAR fusion configurations. Central to our methodology is a novel mixture stack modality training scheme, allowing the map decoder to learn from camera-only, LiDAR-only, and fused features. We also introduce a latent-space projector module to align bird’s eye view features from different modalities into a shared representation space, facilitating feature alignment and improving representation learning. During inference, our model utilizes a switching modality strategy to adapt seamlessly to different evaluated input configurations, including camera-only, LiDAR-only, and camera-LiDAR fusion settings. To evaluate the robustness of HD map construction methods, we designed 13 different sensor corruption scenarios and conducted extensive experiments comparing Uni-Map with state-of-the-art methods. Experimental results show that Uni-Map outperforms previous methods by a significant margin across both normal and corrupted modalities, demonstrating superior performance and robustness.