DRMNet: More efficient bilateral networks for real-time semantic segmentation of road scenes

Abstract Semantic segmentation is crucial in autonomous driving because of its accurate identification and segmentation of objects and regions. However, there is a conflict between segmentation accuracy and real-time performance on embedded devices. We propose an efficient lightweight semantic segmentation network (DRMNet) to solve these problems. We use a lightweight bilateral structure to encode semantic and spatial paths and cross-fuse features during encoding, we also add unique skip connections to coordinate upsampling in the semantic path. We design a new self-calibrated aggregate pyramid pooling module (SAPPM) at the end of the semantic branch to capture more comprehensive multi-scale semantic information and balance the extraction and inference speed of the semantic branch. Furthermore, we designed a new feature fusion module, which guides the fusion of detail features and semantic features through attention perception, alleviating the problem of semantic information quickly covering spatial detail information. Experimental results on the CityScapes and CamVid datasets demonstrate the effectiveness of DRMNet. On a 2080Ti GPU, our model achieves 78.6% mIoU and 78.9% mIoU on CityScapes and CamVid, respectively, with 88.3 FPS and 149 FPS speeds. These results highlight the model's ability to better balance accuracy and real-time performance, making it suitable for embedded devices in autonomous driving applications.