A novel pansharpening: when multiscale adaptive attention meets dual enhanced networks and low-rank tensor decomposition

Deep learning (DL) has achieved advanced results in pansharpening remote sensing images. However, existing DL-based methods still have several drawbacks, such as incomplete feature extraction and insufficient information fusion. In this context, we propose a multiscale adaptive attention pansharpening network that cooperates with dual-enhanced networks and low-rank tensor decomposition. First, we designed dual-enhanced networks: spectral–spatial transformation networks (SST-Net) and multiscale hybrid attention networks (MHA-Net). SST-Net can learn spectral information from low-spatial resolution multispectral images and spatial information from panchromatic images. Moreover, MHA-Net can learn global and local details from a multiscale perspective. Second, to further extract the spectral information from the detail mapping, low-rank tensor decomposition was used to divide the detail mapping into low-rank and sparse parts. The low-rank part corresponds to spectral information, and the sparse part can enhance spatial information. Finally, when generating residual information, we used a multiscale adaptive attention network (MAA-Net) as the backbone to effectively fuse the spatial and spectral information. The experiments were performed on three public pansharpening datasets. The results show that the proposed method achieves results that are superior to those of the conventional and DL-based pansharpening methods.