Detection Drives An End-to-End Fusion of Infrared and Visible Images Based on Diffusion Models

Infrared and visible image fusion methods have shown promising results, yet existing approaches either compromise downstream detection performance through independent fusion processes or sacrifice computational efficiency and flexibility by requiring joint training of fusion and detection models. To address these challenges, we propose a detection-driven image fusion network based on diffusion models (termed as DDIF), which optimizes the fused images specifically for object detection tasks. Our method features the following three aspects: 1) We reformulate the image fusion process as an inverse problem solved by a non-differentiable optimization process wherein the fused result preserves the source modality information while conforming to the image prior provided by the diffusion model. 2) We design a Response Guide Learning Module (RGLM) to learn response maps, which determine the contribution of each modality in the fusion process according to the downstream detection task. 3) We establish explicit gradient relationships to ensure compatibility between RGLM training and the non-differentiable optimization process, enabling end-to-end training. Notably, a moderate coupling mechanism is formed in our frame-work as the subsequent detection model is pre-trained and frozen, enabling flexible integration with various advanced detection networks while maintaining computational efficiency. Extensive experiments indicate that our method achieves superior detection performance compared to SOTA approaches and produces high-quality image fusion results.