NSCT decomposition and VGG-Net based multimodal Infrared and Visible Image Fusion

A novel technique for infrared and visible image fusion is proposed, combining the Non-subsampled Contourlet Transform (NSCT) with VGG-Net by effectively preserving edge details, contrast, and brightness. The image fusion process initiates with the NSCT decomposition of original images yielding low frequency (LF) and band-pass frequency (BF) sub-band coefficients. An energy-based fusion rule merges LF sub-band coefficients. The BF sub-band coefficients undergo processing in the VGG-19 Net to extract deep features. Using l1-norm and a block-based average operator, various candidates of fused BF sub-bands are generated. A maximum selection strategy determines the combined BF sub-band coefficients. The final step reconstructs the amalgamated image through inverse NSCT, ensuring an effective fusion. The proposed method outcomes are better and its objective and visual evaluation is done with five conventional methods and metrics such as Qabf, Entropy, FMI_Pixel, MS_SSIM, MI, SSIM are used for performance evaluation.