Medical image fusion based on deep neural network via morphologically processed residuals

• Proposed a deep neural network-based medical image fusion method that uses morphologically processed residuals to better preserve crucial features from multimodal images like CT, MRI, PET, and SPECT scans. • Introduced a multi-scale image pyramid strategy combining Laplacian and Gaussian pyramids with an adaptive local similarity-based fusion mode selection, enhancing human visual perception consistency in fused images. • Implemented an edge-preserving morphological processing to selectively enhance significant regions while minimizing noise, block effects, and artifacts, maintaining sharpness and contrast in critical image areas. • Achieved superior quantitative and qualitative results over nine benchmark methods (like DFWT, CNN-CP, MATR, DFENet) across various fusion quality metrics (EPI, MI, QAB/F, SF, TE) using CT-MRI, MRI-SPECT, and MRI-PET datasets. • Demonstrated robustness and scalability by reducing network complexity (no fully connected layers) for efficient memory usage and faster computations, while still achieving better fusion performance compared to recent deep learning models. Medical image fusion enhances the intrinsic statistical properties of original images by integrating complementary information from multiple imaging modalities, producing a fused representation that supports more accurate diagnosis and effective treatment planning than individual images alone. The principal challenge lies in combining the most informative features without discarding critical clinical details. Although various methods have been explored, it remains difficult to consistently preserve structural and functional features across modalities. To address this, we propose a deep neural network–based framework that incorporates morphologically processed residuals for competent fusion. The network is trained to directly map source images into weight maps thereby overcoming the limitations of traditional activity-level measurements and weight assignment algorithms, and enabling adaptive and reliable weighting of different modalities. The framework further employs image pyramids in a multi-scale design to align with human visual perception, and introduces a local similarity–based adaptive rule for decomposed coefficients to maintain consistency and fine detail preservation. An edge-preserving strategy combining linear low-pass filtering with nonlinear morphological operations is used to emphasize regions of high amplitude and preserve optimally sized structural boundaries. Residuals derived from the linear filter guide the morphological process ensuring significant regions are retained while reducing artifacts. Experimental results demonstrate that the proposed method effectively integrates complementary information from multimodal medical images while mitigating noise, blocking effects, and distortions, leading to fused images with improved clarity and clinical value. This work provides an advanced and reliable fusion approach that contributes substantially to the field of medical image analysis, offering clinicians enhanced visualization tools for decision-making in diagnosis and treatment planning.