A transformer-based semantic segmentation framework for remote sensing images via spatial-channel cross decoding

Deep learning has become central to recent advances in segmenting remote sensing imagery due to its powerful feature extraction capabilities. Most existing deep learning-based approaches concentrate on spatial or channel-wise feature extraction in isolation. However, overlooking the interplay between these two aspects will significantly impair the performance of semantic segmentation. Motivated by this observation, this paper proposes a Spatial-Channel Cross-Decoding Transformer model (SCCFormer). To effectively extract spatial and channel features from remote sensing images, SCCFormer introduces a dual-stream Transformer decoder network based on a deformable attention mechanism that captures the contextual dependencies of spatial and channel features. Regarding the correlation between spatial and channel features, SCCFormer proposes a multi-stage, spatial-channel features cross-decoding method, which cross-decodes and fuses spatial and channel features at various feature scale levels, thereby capturing the relevant semantic information of both spatial and channel features. The proposed SCCFormer is evaluated based on the Potsdam, Vaihingen, LoveDA and Multi-Source Satellite Imagery for Segmentation remote sensing datasets which demonstrated the effectiveness of the model in the task of remote sensing image semantic segmentation.