A multimodal deep learning framework with contrastive learning and multi-instance learning for endometrial cancer preoperative risk stratification

Abstract Endometrial cancer (EC) risk stratification currently relies on postoperative pathology, limiting preoperative surgical planning. Magnetic resonance imaging (MRI) is the preferred modality for preoperative EC evaluation. However, current research on EC preoperative risk stratification has certain limitations: traditional radiomics demonstrate deficient performance while relying on manual segmentation, and unimodal designs that fail to fully leverage clinical data. To address this, we developed an automated deep learning framework integrating multiparametric MRI and MRI reports from 2,662 patients across six centers. Key innovations include: contrastive learning leverages MRI reports to supervise and refine the image encoder, aligning visual features with diagnostic semantics; and multi-instance learning dynamically aggregates features from multi-parametric MRI sequences, even in the presence of missing data. Our multimodal model significantly outperformed unimodal approaches, achieving AUCs of 0.827 (internal validation) and 0.768-0.863 (four external cohorts), improving AUC by 4.7%-10.2% versus image-only and 4.1%-8.5% versus text-only models. This demonstrates strong generalizability and offers potential to optimize surgical planning, improve prognosis, and reduce complications.