A semantic segmentation method for road scenes based on small samples

Previous convolutional neural networks (CNNs) in the field of semantic segmentation, particularly in road scenes, suffer from overfitting, insensitivity to positional information, and limited robustness due to convolution and pooling operations. In this paper, we propose a multi-scale multi-feature fusion self-attention network (MSMA-Net) based on the U-Net architecture. The Decoder stage of the U-Net structure is removed, retaining only the first four sampling layers of the Encoder. The final features from each layer are simultaneously fed into a multi-scale pyramid pooling structure, where different scale pooling operations merge the features into a unified dimension. The output features are then passed through a Transformer Encoder stage and the MLP Head to produce the final classification results. The proposed method is trained on the Cityscapes and CamVid datasets, with half of the data randomly selected for training. The achieved average intersection over union (mIoU) scores are 77.9% and 72.2% on Cityscapes and CamVid, respectively, demonstrating significant advantages over other networks.