An adaptive spectral-temporal feature learning Graph-Transformer network for land cover mapping with optical time series

Land cover mapping with optical time series plays an important role in the study of urban expansion and farmland conservation. Due to the relationships among spectral responses of ground objects in different bands and at different times lack a clear functional representation, classification models often fail to fully mine and utilize the fine-grained change information of the objects over time and in different spectral bands, resulting in great challenges in improving the accuracy of land cover mapping. We propose a dual-branch spectral-temporal adaptive learning Graph-Transformer network to discover the temporal and spectral patterns of optical time series for land cover mapping. (1) Constructing a graph learning module to adaptively learn the relationships of different bands and using the quantitative relationships to construct an adaptive graph network, so as to iteratively learn the fine-grained spectral responses of different objects using a graph convolutional network with mix-hop propagation; (2) adopting the Transformer with long time-series dependence and correlation extraction capabilities to extract the fine-grained change information of different ground objects over time; (3) cross-fusing the discriminable spectral-temporal features and realizing high-precision classification through a deep feature enhancement module. The experimental results on the benchmark dataset Breizhcrops, ILI and selected Sentinel-2 time series show that using the proposed Graph-Transformer network, the accuracy of land cover mapping reached 81.30%, 95.51% and 97.00%, respectively, outperforming state-of-the-art approaches. It is proved that the proposed spectral-temporal adaptive learning strategy is effective in improving the classification accuracy of remote sensing time series.