Unification of image fusion and super-resolution using jointly trained dictionaries and local information contents

Multi-modal imaging requires image fusion to combine advantages of different types of sensors and requires super-resolution (SR) because of limited spatial resolution of source images. In this study, a novel framework is proposed for unification of image SR and the fusion process to obtain a high-resolution (HR)-fused image from a set of low-resolution (LR) multi-modal images. The jointly trained dictionaries of LR patches and corresponding HR patches are used for sparse representation of LR source image patches and HR-fused image patches, respectively. The sparse coefficients vectors for corresponding patches of source LR images are determined by using orthogonal matching pursuit and a local information content-based metric is employed to fuse these sparse coefficients. The corresponding HR-fused image patch is obtained by combining elements of the HR dictionary as per the fused coefficients of the LR image patches. The experimental results on sets of multi-modal images exhibited that the proposed method outperformed the existing fusion and SR techniques in terms of visual quality and image fusion quality metrics.