Machine Learning for Radiomics in Oncology: Challenges, Limitations, and Future Directions

In precision oncology, the combination of the strengths of both histopathology and medical imaging provides a fertile ground for tumor characterization. Although histopathology offers a definitive cellular diagnosis, this approach is invasive and only provides a small-scale characterization of the tumor, while medical imaging modalities, such as X-ray, CT, MRI, ultrasound, and PET scans, provide a complete characterization of the tumor but, until recently, relied on the subjective ability of a human observer. The application of machine learning to radiomics aims at filling this gap, as images are mined to reveal patterns of disease not visible to the naked eye. In this perspective paper, the trajectory of machine learning in radiomics for oncology applications is critically discussed. By exploring studies using different imaging modalities, we seek to look beyond the achievements of innovative algorithms and identify the systemic weaknesses in the field, which are holding it back from translating to the clinic. In this regard, we identify two major challenges in the field: the significant effects of inter-modality and inter-scanner variability in model generalizability, and the ‘interpretability gaps’ in understanding the rationale for the decision-making process in ML algorithms. In this paper, we assert that these challenges are holding back even the best of algorithms and thus set the direction for the field in the future, advocating for the development of ML systems with emphasis on their performance in real-world settings as opposed to the lab.