Spatial-Temporal Semantic Feature Interaction Network for Semantic Change Detection in Remote Sensing Images

This is the authors' abstract. We don't add key points for this paper.

Semantic Change Detection (SCD) in Remote Sensing Images (RSI) aims to identify changes in the type of Land Cover/Land Use (LCLU) corresponding to changed areas in RSI. The “from-to” information of the acquired image has more profound practical significance than Binary Change Detection (BCD). However, most deep learning-based SCD algorithms do not fully exploit the spatial-temporal information of multi-level features, leading to challenges in extracting LCLU features in complex changing scenes. To address these issues, we propose a Spatial-Temporal Semantic Feature Interaction Network (STS-FINet) to improve the performance of SCD in RSI. The proposed STS-FINet comprises a Multi-Scale Feature Extraction Encoder (MS-FEE), a Transformer-based Multi-Level Feature Interaction module (TML-FI), and a Multi-Level Feature Fusion Decoder (ML-FFD). The MS-FEE extracts deep semantic and differential information from the RSI. The TML-FI is designed to mine the spatial-temporal information by extracting long-range dependencies and spatial information from multi-level features to improve spatial perception. Moreover, Mixed Spatial Reasoning Convolution block (MixSrc) is presented to enrich the spatial information by extracting the multi-scale features, thus improving the model's capability to interpret complex changing scenes. Finally, ML-FFD integrates the multi-level features, resulting in the generation of the semantic change map. The effectiveness of the proposed STS-FINet is verified on two high-resolution RSI datasets. Experimental results show that the proposed STS-FINet achieves better change detection performance than SOTA methods. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/YuhangZh/STS-FINet</uri>.