MIMO-TGAN: Multi-Modality Medical Image Fusion via Triple Generator Network for Brain Abnormality Detection

Medical image fusion (MIF) significantly improves diagnosis accuracy by combining complementary data from several imaging modalities. However, the conventional fusion methods often fail to maintain the edge details, uniformity of texture and most relevant characteristics. To overcome these challenges, a novel multi-modality medical image fusion using Triple generator network (MIMO-TGAN) is proposed for accurate brain abnormality detection. The proposed MIMO-TGAN uses the multi-modality images such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT), and Positron Emission Tomography (PET) images for MIF process. Initially, the gathered MRI and CT images are denoised with scalable range-based adaptive trilateral (SCRAT) filter and PET images are processed by multi-scale retinex (MSR) technique. The proposed MIMO-TGAN model contains three generators with a discriminator and an abnormality detection phase. The Generator-1 consists of grouped convolutional with different attention blocks for retrieving the relevant features from MRI and CT images to generate a synthesized image. Simultaneously, Generator-2 retrieves the most important voxel features from PET images and MRI images to generate another synthesized image. Finally, Generator-3 is designed to combine the both synthesized images from two generators by using four fusion rules to accurately detect brain abnormalities. The experimental evaluations demonstrate that the proposed MIMO-TGAN achieves an accuracy of 99.43% and 98.24% for dataset-1and dataset-2 respectively. Moreover, the proposed MIMO-TGAN achieves the entropy values of 7.05 and 6.84 for dataset-1 and dataset-2, and overall SSIM of 0.94 that indicates superior fusion quality results compared to existing methods.