Sensor Fusion and Motion Planning with Unified Bird’s-Eye View Representation for End-to-end Autonomous Driving

End-to-end autonomous driving has made significant advancements in these years. Efficiently fusing multi-modal sensor information to enhance the scene understanding capability and motion planning performance of end-to-end models is currently a prominent research topic. Existing methods fuse multimodal information from different views, which lack efficiency and restrict the sensor extension of the models. In this work, we propose an end-to-end autonomous driving method with unified multi-modal feature representation. Construction of the entire end-to-end autonomous driving model, including sensor fusion and motion planning, is conducted in a unified Bird’s-Eye View(BEV) representation space. We summarize our proposed method into three stages: multi-modal BEV feature construction, multi-modal BEV feature fusion and motion planning in BEV space. This construction method significantly improves the environmental perception capability and planning performance of the end-to-end model. We conduct closed-loop experiments in the Carla simulator and our method achieves superior performance compared with other methods.