MACCNet: Multiscale Attention and Cross-Convolutional Network for Infrared and Visible Image Fusion

Infrared and visible image fusion (IVIF) aims to fully preserve the target and detail information of the infrared and visible images in the fusion image. Although deep learning–based methods have been widely used in IVIF, they usually use the same network structure to extract features without considering the differences between different image modalities, leading to insufficient feature extraction and unsatisfactory fusion results. To overcome these problems, we propose a multiscale attention and cross-convolution network (MACCNet) to obtain competitive fusion results. The method includes a new two-branched structure-based encoder network for extracting features from two different modality images. In one branch, a new multiscale attention module (MAM) extracts the target features at different scales of the input infrared images. In the other branch, the new cross-convolution feature extraction module (CFEM) extracts the detail features of visible images in different directions. We also introduce a local saliency attention fusion network (LSAFN) to obtain two weight maps to improve the fusion of extracted target and detail features of the different modality images. Additionally, the two weight maps are averaged as coefficients of the pixel loss terms to adaptively train the network. Finally, we obtain the final fusion result by reconstructing the fused features via a decoder network. Experimental results show that the proposed MACCNet outperforms several state-of-the-art IVIF methods in terms of visual perception and objective evaluation.