Benchmark study demonstrates performance degradation of autonomous vehicle localization algorithms on complex routes, highlighting vulnerabilities in current navigation models.
Key Points
To establish a diverse, multi-sensor dataset across challenging real-world routes to evaluate the robustness and generalization of modern autonomous driving localization and odometry algorithms.
Collected 60 driving sequences spanning 643 km across 9 real-world routes, with repeated passes under varying traffic and weather conditions.
Equipped a test vehicle with a 5MP camera, 360-degree Doppler radar, 128-channel lidar, FMCW Doppler lidar, IMU, wheel encoders, and centimetre-level GNSS-INS ground truth.
Evaluated state-of-the-art odometry and metric localization algorithms via an open-source development kit and public leaderboard.
State-of-the-art odometry and localization algorithms degraded significantly on the complex Boreas-RT routes compared to standard benchmarks.
Repeated traversals confirmed that existing algorithms overfit to simple driving environments and lack robustness against diverse environmental variations.