Sliced-Wasserstein distance guided unsupervised domain adaptation for semantic segmentation of remote sensing images

Learning domain-invariant discriminative features is fundamental in unsupervised domain adaptive semantic segmentation (UDASS). For the remote sensing image semantic segmentation task, the same class probably varies across domains, whereas different classes may show similar features, which makes the domain-invariant features hard to learn. To address the challenges, the sliced-Wasserstein distance guided feature alignment network (SWDFAN) was proposed for the UDASS of remote sensing imagery. The SWDFAN utilizes an encoder–decoder architecture, in which the encoder is based on Swin Transformer, and the decoder uses the proposed multi-scale and multi-level fusion (MSLF) module. Specifically, the MSLF module, integrating the spatial and channel squeeze and excitation module and the context-aware fusion module, is designed to capture the fine-grained class-specific features. In addition, a sliced-Wasserstein Constraint method is proposed to reduce the discrepancy of class feature distributions, which enhances the alignment of intra-class features and the separation of inter-class features. Moreover, a new dataset, the Xiaoshan-Binjiang (XB) dataset, is introduced for UDASS of remote sensing images. In addition to the XB dataset, the LoveDA dataset was also employed to validate the effectiveness of the proposed method. The proposed method achieves a 3.82% improvement in mean intersection over union (mIoU) over state-of-the-art approaches on the XB dataset. Furthermore, on the urban-to-rural and rural-to-urban tasks of the LoveDA dataset, the proposed method demonstrates mIoU improvements of 0.45% and 1.94%, respectively, compared with the previous methods.