The Utilization of DNN-based Semantic Segmentation for Improving Low-Cost Integrated Stereo Visual Odometry in Challenging Urban Environments

Positioning and Navigation (PN) is one of the most important topics in the world of Autonomous Vehicles (AVs). Being equipped with a suite of sensors and high-performance computers, self-driving cars are designed to perceive its surrounding environment prior to planning and control. Among the observations are the semantics of the objects appearing in the scene. While PN is very challenging for extended GNSS outages, vision sensors can also exhibit failures in the case of highly dynamic scenes and lack of texture between consecutive image frames. To overcome this problem, we propose a stereo visual odometry scheme and advocate the use of a pretrained state-of-the-art Semantic Segmentation (SS) Deep Convolutional Neural Networks (CNN) model to forcefully remove features belonging to objects that most likely behave dynamically in the scene prior to egomotion estimation and integration with inertial sensors. When loosely coupled with inertial sensors, the proposed method was able to outperform the integrated algorithm without SS-based outlier rejection during natural GNSS outages.