Deformable multi-head cross attention-based 3D U-net for context-aware multi-modal cardiac image segmentation with uncertainty-aware maps

Highly efficient medical image segmentation is crucial for accurate tumor diagnosis in multi-modal imaging datasets having Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) images. Nevertheless, current algorithms have limitations while incorporating supplementary data from multiple modalities, which affects the segmentation performance and interpretability. This proposed study uses a 3D U-Net for hierarchical feature extraction. The 3D U-Net improves segmentation performance by strengthening the model’s capacity to capture local and global relationships. The Swin transformer used in the proposed work helps in context-aware reasoning. This work suggests a deformable Multi-Head Cross-Attention mechanism (MHCA) for effective feature fusion. The uncertainty-aware map and SHAPely-based explainability help the experts in decision-making. The proposed system has demonstrated good performance during the experimental evaluation with multiple datasets. The proposed method achieves a Dice Coefficient of 91.8% on Medical Decathlon (MD), 92.3% on Automated Cardiac Diagnosis Challenge (ACDC), and 93.2% on Multi-Modality Whole Heart Segmentation (MMWHS) datasets, overcoming the baseline models.