SSDFusion: A Semantic Segmentation Driven Framework for Infrared and Visible Image Fusion

Abstract By fusing infrared images with visible images, it is possible to obtain more abundant and accurate information content, thereby enhancing the accuracy and application value of image information. However, existing infrared and visible image fusion methods often lack attention to the semantic information and global context information in the original images. To address these issues, we propose a novel deep learning framework for infrared and visible image fusion , which is named Semantic Segmentation Driven Infrared and Visible Image Fusion Framework (SSDFusion). Within the fusion framework, the Local Global Feature Extraction Fusion Module is employed, complemented by the decoder. Furthermore, under the guidance of semantic segmentation, SSDFusion achieves a better understanding of complex scene region information, enhancing fusion task performance. Finally, an adaptive loss function is implemented throughout SSDFusion to fine-tune the balance between the semantic segmentation task and the image fusion task by adjusting their proportional contribution. This approach aids in more accurately preserving the semantic information in the image, thereby enhancing the performance of the fusion framework. SSDFusion was evaluated on the MSRS and RoadScene datasets, and the results show that our approach exhibited better performance in many aspects.