Development and Validation of a Multimodal-Multitask Deep Learning Approach for Estimating Late Distant Recurrence Risk in HR–Positive Early Breast Cancer

Late distant recurrence (DR) remains a persistent risk in hormone receptor–positive (HR+) early breast cancer after completion of 5 years of endocrine therapy. We developed and validated a multimodal artificial intelligence (AI) model to improve long-term risk stratification and to explore heterogeneity in benefit from extended letrozole therapy (ELT). The deep learning model integrating digitized hematoxylin and eosin whole-slide images with clinicopathologic variables was developed using 2,271 patients from the NSABP B-42 trial with 5-fold cross-validation and externally validated in 4,300 patients from the TAILORx trial who were disease-free at 5 years from initial diagnosis. Prognostic performance was evaluated using hazard ratios (HRs) and absolute risk differences. Exploratory analyses assessed extended letrozole therapy (ELT) benefit across model-defined risk groups. In NSABP B-42, the model stratified patients into groups with markedly different outcomes, with a 10-year absolute DR risk difference of 7.95% between high- and low-risk groups (HR, 5.71; 95% CI, 3.5–9.317; P<0.001). High-risk patients derived greater absolute benefit from ELT (4.09%) than low-risk patients (0.49%). External validation in the independent TAILORx cohort confirmed prognostic performance, with MI Clarity M3T identifying patients with significantly different late DR outcomes (HR, 1.893; 95% CI, 1.413–2.534; P<0.001). This multimodal AI approach using routine pathology and clinical data enables robust and generalizable stratification of late distant recurrence risk in HR+ breast cancer. This scalable strategy may complement existing genomic assays and support more individualized decisions regarding extended endocrine therapy.