A Performance Evaluation of Open Source Autonomous Driving Frameworks: Case Studies of Apollo and Autoware

Autonomous vehicles (AVs) are revolutionizing transportation by enhancing safety and efficiency. Despite significant advancements, the literature lacks a systematic benchmarking methodology for open-source autonomous driving stacks (ADS) in realistic simulation environments. This paper presents a methodology for comparing the performance of different autonomous driving (AD) frameworks. It has been applied to compare two leading open-source frameworks, Autoware.AI and Apollo.Auto, focusing on their trajectory planning modules in simulated scenarios. We conducted comprehensive simulations using the LGSVL simulator, integrating both stacks with high-fidelity sensor models and realistic environments. Using a diverse set of performance metrics, including trajectory deviation, computational effort, safety distance, energy consumption, and trajectory smoothness, we could quantitatively assess any planner. Our assessment revealed distinct trade-offs between the planners. Autoware.AI’s Open Planner provided higher safety margins by maintaining larger clearance distances from obstacles, resulting in increased path lengths and energy consumption. Conversely, Apollo.Auto’s Public Road planner offered smoother trajectories and enhanced passenger comfort but required higher computational resources. This work bridges the gap in benchmarking open-source ADS, providing valuable insights into their performance. Our findings contribute to the development of safer and more efficient autonomous vehicles by offering a systematic evaluation of key open-source platforms and highlighting the trade-offs inherent in trajectory planning.