Dual Directional Complementary Gradient Fusion and Deep Refinement for Hyperspectral Image Super Resolution

The spatial and spectral resolution trade-off in the hyperspectral imaging is a fundamental and essential issue, and automatically generating high-resolution images in both spatial and spectral domains (HR-HS) by merging a low spatial resolution hyperspectral (LR-HS) image and a high spatial resolution RGB (HR-RGB) image, which are captured by the existing commercial sensors, has recently attracted extensive attention. Motivated by the powerful representation capability of the deep learning networks, current dominated methods have devoted to design deep and complicated network architectures, and manifested great performance progress. This study aims to exploit a simple yet effective deep model by aggregating the complementary missing information into the feature learning branches and automatically modeling the relationship between the target and observations. Specifically, we incorporate the spectral gradient of the LR-HS image with the feature learning branch of the HR-RGB image while aggregate the spatial gradient of the HR-RGB image into the learning branch of the LR-HS image to enhance the representation capability in both spatial ans spectral domains. Moreover, we reconstruct a serials of target HR-HS image from the fused features of dual branches, and employ the un-recovered residuals in the observations by automatically learning the degradation procedure to further refine the former reconstruction in an asymptotic way. Comprehensive experiments have demonstrated that our proposed deep model for HR-HS image reconstruction achieves superior SR performance over state-of-the-art methods in term of quantitative metrics and perceptive quality.