Semantic-Aware Autonomous Vehicle Navigation with Causal Transformers and Q-Learning

Visual perception is crucial for autonomous vehicle navigation, enabling situational awareness through depth estimation and semantic segmentation. This study presents a novel transformer-based pipeline for depth perception and semantic aware path planning for enhanced visual understanding. Our approach combines a dense prediction model for relative depth estimation and semantic information with a causal transformer and Q-learning for navigation. We evaluate the pipeline’s performance using metrics such as Root Mean Square Error and Depth Accuracy alongside safety-based indicators. Qualitative assessments of agent behavior in diverse simulated environments and real-world scenarios demonstrate improvements in autonomous vehicles’ contextual understanding of surroundings. Our results demonstrate that the proposed pipeline enhances obstacle anticipation and vehicle responsiveness in simulated and real-world environments. By improving the reliability of autonomous navigation systems, this work contributes to SDG 9 by advancing resilient infrastructure for intelligent transportation while supporting SDG 11 through safer and more efficient urban mobility solutions.