Two-stream Maximal Feature Attention-guided Contrastive-learning GAN for Image Fusion

Off-the-shelf GAN-based image fusion methods solely utilize the diverse modal images as reference samples, aiming to make the distribution of the generated image closely resembles that of the source images through the adversarial process. This leads to the resultant images that resemble the compromised outcomes of visible images and infrared counterparts. This article reports a seminal study on fusing infrared and visible images, which ingeniously introduces the maximal feature constraints built upon contrastive learning into GAN-based image fusion framework, facilitating more effective semantic knowledge extraction from both image types. The generator is devised as a lateral feature aggregation codec structure, aiming to enhance interaction among multi-level features and compensate for feature loss during transmission. Unlike methods that focused only on edge information, a novel two-stream maximal feature-guided attention network upon the multi-level exclusive features is designed to strengthen the attention towards prominent information during fusion. This allows finer texture detail restoration and highlighted targets maintaining while attenuating noise. A hybrid loss function, defined as a combination of content loss and adversarial-perceptual-contrastive constraint, is minimized to generate fusion results with source imagery statistics by employing an objective that concentrates on feature distribution rather than simply evaluating pixel intensity difference. Extensive trials demonstrate that the proposed algorithm performs better than previous top-performing methods in both visual and objective aspects on four benchmark datasets.