Key points are not available for this paper at this time.
We developed a multimodal dataset that combines features computed from radiomics, clinical data, medical and histopathology images routinely collected to predict clinical outcomes for NSCLC patients. Patients with unresectable LA-NSCLC were enrolled in a prospective trial and underwent concurrent definitive radiochemotherapy (CRT) with an adaptive approach from 2012 to 2014. CT scan, clinical data, histopathology slides were evaluated.Semantic features age, sex and smoking habit, TNM stage and histology and mutations; 2) Radiomic feature extracted from 3D ROIs given by Clinical Target Volume (CTV) of CT scans performed before starting concurrent CRT; 3) Pathomics features: hystological slides were digitized obtaining whole slide images (WSI) loaded on QuPath software and re-evaluated by an expert lung pathologist that selected representative number of tumor areas and in these areas were segmented the tumor edge avoiding fibrosis, necrosis or histological artifacts. Different multimodal late fusion rules and two patient-wise aggregation rules leveraging the richness of information given by CT images, whole-slide scans and clinical data were analyzed. Overall 50 patients with locally advanced NSCLC were enrolled in the adaptive protocol. Among these patients, 35 patients had available histological slides and were included in this analysis. In this latter group, 13 (37.1%) patients achieved a significant reduction during chemoradiation. We investigate eight different multimodal late fusion rules and two patient-wise aggregation rules in comparison to unimodal approach. The experiments showed that the proposed fusion-based multimodal paradigm, achieving an AUC equal to 90.9%, outperforms each unimodal approach, suggesting that data integration can advance precision medicine. The combination of the three modality resulted in the highest performance of the model. We propose a deep learning framework for integrating multimodal data for prediction of early response to treatment. This approach has the potential to advance clinical decision support systems, enable personalized medicine, and could represent an effective and innovative step forward in personalised medicine.
Greco et al. (Fri,) studied this question.