Artificial Intelligence-Assisted Risk Stratification in Stage II Colorectal Cancer: Multi-Institutional Validation of Semantically Enhanced Deep Learning

BACKGROUND & AIMS: Accurate risk stratification in Stage II colorectal cancer is essential for treatment decision-making, as current guidelines recommend adjuvant chemotherapy only for patients with a high risk of relapse. We aimed to develop and validate an artificial intelligence-based approach for automated invasive front assessment to improve prognostic stratification in this population. METHODS: We developed Semantically-Enhanced Multiple Instance Learning (SÉMIL), integrating vision-language foundation models with attention-based multiple instance learning for automated invasiveness assessment from H&E-stained whole slide images. We trained and validated SÉMIL on 1608 H&E-stained whole slide images from 3 cohorts (Austin n = 697, MCO n = 478, DYNAMIC n = 433). We compared SÉMIL performance against manual pathologist assessment and nonsemantic MIL approaches. RESULTS: For binary classification, SÉMIL outperformed nonsemantic MIL across all cohorts (external validation: AUC 0.713-0.821 vs 0.686-0.803). For survival prediction, SÉMIL demonstrated validated prognostic stratification in both the internal (Austin: hazard ratio [HR] = 4.73; P = .0012) and the 2 external (MCO: HR = 2.84; P = .0032; DYNAMIC: HR = 2.10; P = .0396) Stage II validation cohorts. Critically, among National Comprehensive Cancer Network guideline-defined high-risk Stage II patients, SÉMIL successfully stratified outcomes across all 3 cohorts (HRs, 2.96-3.50; all P < .05), demonstrating consistent reproducible performance. In multivariate analysis of the combined Stage II cohort (n = 1220), SÉMIL retained independent prognostic significance (HR = 1.98; P = .005) after adjusting for conventional clinicopathologic features including T stage, MMR status, and lymph node examination adequacy. Concordance analysis between SÉMIL and manual assessment showed concordant infiltrative classification identified the highest-risk group (HR = 3.96; P < .0001), with discordant cases showing intermediate risk. CONCLUSIONS: SÉMIL demonstrates validated prognostic stratification in Stage II colorectal cancer, with potential utility for refining risk assessment within National Comprehensive Cancer Network guideline-defined high-risk categories where treatment decisions are most challenging.