A novel deep multimodal contrastive network for early prediction of cognitive deficits using multimodal MRI and clinical data

Deep learning has shown promising results in early predicting cognitive deficits in very preterm infants using multimodal brain MRI data, acquired soon after birth. Prior methods focus on the feature fusion of different modalities but ignore the latent high-order feature similarity information. In this study, we propose a novel deep multimodal contrastive network using T2-weighted structural MRI (sMRI), diffusion tensor imaging (DTI), resting-state functional MRI (rsfMRI), and clinical data to predict later cognitive deficits. Our proposed model significantly improved predictive power compared to other peer models for early diagnosis of cognitive deficits.