PSSTFN: A Progressive Spatial–Temporal–Spectral Fusion Network for Remote Sensing Images

Spatial-temporal-spectral fusion (STSF) is highly desirable to generate dense-time image series with high spatial and spectral resolution by integrating the complementary advantages of multi-source and multi-temporal observations. However, most existing STSF methods are still limited to the assumption of linear temporal, spatial and spectral relations. In addition, the STSF methods on Landsat and MODIS data are insufficient to characterize the inherent properties of the current spaceborne hyperspectral images with low spatial and temporal resolutions. For these, we propose a progressive STSF network (PSSTFN) by interestingly integrating spatial-spectral fusion and spectral-temporal fusion into a unified end-to-end STSF framework. Specifically, in the spatial-spectral fusion stage, we obtain the hierarchical features with different receptive fields and propose a multi-attention guided module for joint learning and refinement of spatial-spectral features. In the spectral-temporal fusion stage, a feature insertion module is presented to embed the difference images into the resulting spatial-spectral features, and the estimation from deeper layers is cascaded for more reliable spatial information. We build Dongying (DY) and Yellow River Estuary (YRE) remote sensing datasets based on Sentinel-2 and ZiYuan(ZY)-1 02D satellites for verification, and the experimental results on reduce- and full-resolution data demonstrate the superior performance of our method over the existing methods visually and quantitatively.