Hierarchical Blending with Transformer for Infrared and Visible Image Integration

Introduction: Infrared-visible (IR-VIS) image blending aims to synthesize a comprehensive representation by integrating complementary information from heterogeneous modalities. This research addresses the limitations of existing deep learning methods in harmonizing the extraction of multi-receptive-field representations, modeling long-short range dependencies, and detail- preserving recovery within a unified framework for IR-VIS image blending. Methods: We propose HBFormer, a novel hierarchical Transformer-based architecture. Key methodological innovations include: 1) the Dense Multi-Receptive-Field Block (DMRFB) with Ghost-Shuffle convolution for efficient multi-receptive-field representation abstraction, 2) the Bidirectional Attention Cooperation Block (BACB) for parallel short-long range dependency modeling, and 3) the Hierarchical Pooling Tokenization Block (HPTB) for computationally efficient multi-resolution integration. Additionally, a Detail-aware Perceptual Blending Loss (DPBLoss) was designed to enforce edge sharpness, semantic alignment, and modality-specific contrast. The proposed method was evaluated on the TNO and MSRS benchmark datasets. Results: Extensive experiments demonstrate HBFormer’s superiority. On the TNO and MSRS datasets, HBFormer achieved state-of-the-art Mutual Information (MI) scores of 3.62 and 3.54, respectively. These results represent gains of 8.3% and 6.9% over prior Transformer-based methods in MI. Subjective evaluations by expert assessors further confirmed its perceptual excellence, with HBFormer attaining a composite score of 4.7 out of 5. Discussion: The significant improvements in quantitative metrics and perceptual quality highlight HBFormer's effectiveness in systematically unifying multi-receptive-field encoding, attention- guided integration, and structural recovery. These findings suggest that the proposed architectural innovations successfully address critical challenges in IR-VIS image blending, offering a more robust and comprehensive solution compared to existing approaches. While demonstrating state-of-the-art performance, future work could explore further computational optimizations. Conclusion: HBFormer provides a superior framework for IR-VIS image blending by effectively integrating complementary information and preserving critical details. Its advancements in representation learning and attention mechanisms offer significant improvements over existing methods, with strong potential for practical applications requiring high-quality fused imagery.