Hyperspectral Pansharpening Based on Detail Injection and HyperTransformer

Hyperspectral (HS) pansharpening aims to fuse a low spatial resolution HS image with a registered high spatial resolution panchromatic (PAN) image to generate the enhanced HS image with both high spatial and spectral resolution. Existing pansharpening approaches based on convolutional neural networks (ConvNets) are poorly interpretable and rarely use attention mechanisms to extract relevant texture details. In this paper, we present a novel HS pansharpening method based on detail injection framework and HyperTransformer. Firstly, the feature extraction sub-network is used to extract the texture and spectral information of the PAN and HS images, secondly, we designed the spatial-spectral fusion sub-network based on HyperTransformer to obtain the detail information, in which the cross-feature space dependency between PAN and low resolution HS features is extracted and the high-frequency details with more spectrally similar features are learned by the attention mechanism. Finally, the detail injection framework is applied to fuse the detail information and the low-frequency component to obtain the final high-resolution HS image. Comparative experiments conducted on three widely used datasets demonstrate the superiority of the proposed method over the state-of-the-art pansharpening results in terms of both subjective visual effects and quantitative evaluations.