ASGT-Net: A Multi-Modal Semantic Segmentation Network with Symmetric Feature Fusion and Adaptive Sparse Gating

In the field of remote sensing, accurate semantic segmentation is crucial for applications such as environmental monitoring and urban planning. Effective fusion of multi-modal data is a key factor in improving land cover classification accuracy. To address the limitations of existing methods, such as inadequate feature fusion, noise interference, and insufficient modeling of long-range dependencies, this paper proposes ASGT-Net, an enhanced multi-modal fusion network. The network adopts an encoder-decoder architecture, with the encoder featuring a symmetric dual-branch structure based on a ResNet50 backbone and a hierarchical feature extraction framework. At each layer, Adaptive Weighted Fusion (AWF) modules are introduced to dynamically adjust the feature contributions from different modalities. Additionally, this paper innovatively introduces an alternating mechanism of Learnable Sparse Attention (LSA) and Adaptive Gating Fusion (AGF): LSA selectively activates salient features to capture critical spatial contextual information, while AGF adaptively gates multi-modal data flows to suppress common conflicting noise. These mechanisms work synergistically to significantly enhance feature integration, improve multi-scale representation, and reduce computational redundancy. Experiments on the ISPRS benchmark datasets (Vaihingen and Potsdam) demonstrate that ASGT-Net outperforms current mainstream multi-modal fusion techniques in both accuracy and efficiency.