09 | Integration of hematoxylin and eosin and immunohistochemistry images to improve AI-based recurrence prediction in negative sentinel lymphnode melanoma patients

Background: Stage IB–IIC melanoma with negative sentinel lymph node (SLN) status shows heterogeneous recurrence risk. Hematoxylin and eosin (H&E) slides capture tissue morphology, while immunohistochemistry (IHC) assays highlight immune, proliferative, and lymphvascular features. To date, no artificial intelligence (AI)-based model has combined these modalities for recurrence prediction in melanoma.Methods: We analyzed tumor tissues from 68 SLN-negative patients treated at IRCCS Istituto Tumori of Bari (Table 1, 29.4% recurrence rate within 2 years). For each sample, regions of interest (ROIs) were annotated by pathologists on H&E slides, distinguishing TUMOR areas (tumor tissue only) and TUMOR+INF areas (tumor and surrounding infiltrating cells). Corresponding IHC slides, including β-catenin, CD4-CD8, CD20, CD64, CD163, Ki-67, FOXP3, and podoplanin, were co-registered to the H&E reference (Figure 1). From these ROIs, ~3,000 TUMOR tiles and TUMOR+INF tiles (296×296 px at 40× magnification) were automatically extracted. An AI-based approach was implemented using a multi-channel convolutional neural network that jointly processes co-registered stains, capturing pixel-level spatial correspondences. Two distinct models were trained: one for TUMOR ROIs and another for TUMOR+INF ROIs. Performances were evaluated according to a 5-fold cross-validation scheme.Results: Compared with our previous H&E-only approach [1], where AUCs were 0.791 for TUMOR and 0.623 for TUMOR+INF, the integration of IHC features markedly improved predictive performance in both ROI categories. With the multimodal model, the TUMOR-only analysis achieved an AUC of 0.850 (95%CI [0.738–0.945]), while the TUMOR+INF model reached 0.926 (95%CI [0.852–0.979]). The largest performance gain was observed in TUMOR+INF regions, underscoring the added value of immune and microenvironmental features that are not adequately captured by H&E morphology alone.Conclusions: Multimodal integration of H&E and IHC enhances the predictive strength of AI-based models by uncovering insights that are less discernible on morphology alone.References:[1] https://doi.org/10.1186/s12967-024-05629-2 Table 1. Figure 1.