Tumor recurrence or treatment effect? Large multi-institutional evaluation of an AI risk assessment model in glioma and brain metastases.

2080 Background: Distinguishing true tumor recurrence (TuR) from radiation necrosis (RN) on post-treatment MRI scans remains a major neuro-oncology challenge. We hypothesized that an integrated Spatial, Morphologic, and Textural radiomics risk (SMART-risk) model that comprehensively captures local and spatial organization of lesion heterogeneity, can unravel distinct biology across TuR and RN, on clinical MRI; and improve distinction over a data-driven ResNet50 deep learning model. Methods: We retrospectively collected multi-institutional post-treatment MRI cohorts: brain metastases (233 studies; from Cleveland Clinic (CCF), University Hospitals Cleveland (UH), University of Wisconsin (UW); and glioma (340 studies from Indiana University (IU), Dana-Farber Cancer Center, Thomas Jefferson University (TJU), and CCF). Over 80% of the studies were pathologically confirmed as TuR or RN. Institution-held-out external testing was performed (metastases: train CCF+UH, test UW; glioma: train IU+TJU+Dana-Farber Cancer Center, test CCF). Following segmentation, 944 radiomic features/lesion were extracted including graph-based spatial organization of tumor heterogeneity (GrRAiL), local gradient texture heterogeneity (COLLAGE), Haralick, and morphology. LASSO-selected features were integrated in a Random Forest classifier. Performance metrics included cross validation accuracy (CV), test accuracy, F1 score, and AUC; interpretability used SHAP. Comparison was performed with a ResNet50 baseline model. Results: SMART-Risk demonstrated consistent discrimination on the institution-held-out external test set (Table 1) with ~80% accuracy; ~10-15% improvement over a ResNet50 model. SHAP analysis indicated that features corresponding to spatial organization and local heterogeneity, i.e. average path length, node count and entropy measures, were dominant contributors (Mann–Whitney U test, p ≤ 0.001). Recurrent tumors showed higher spatial complexity and greater heterogeneity than RN. Conclusions: SMART-Risk may provide a noninvasive approach to distinguish TuR from RN on routine post-contrast T1-weighted MRI. By integrating spatial-organization (GrRAiL), texture (COLLAGE/Haralick), and morphology features, SMART-Risk captures complementary signatures and may improve discrimination compared with any single feature family. An AI-based SMART-Risk approach may reduce diagnostic ambiguity, support earlier treatment decisions, and avoid unnecessary invasive procedures. Test set performance of SMART-Risk vs ResNet50 for TuR vs RN classification. Cohort (external test site) Model CV accuracy Test accuracy F1 score AUC Metastatic cohort (UW) SMART-Risk 0.78 ± 0.08 0.79 0.78 0.86 Metastatic cohort (UW) ResNet50 0.70 ± 0.09 0.60 0.67 0.63 Glioma (CCF) SMART-Risk 0.83 ± 0.06 0.80 0.84 0.87 Glioma (CCF) ResNet50 0.69± 0.10 0.65 0.77 0.70