The application of machine learning to the multidisciplinary assessment and management of oesophageal cancer

BACKGROUND & AIMS: Rising workflow pressures within oesophageal cancer multidisciplinary teams (MDT) can potentiate inconsistent decision-making, decision-fatigue, and even health inequality. Machine learning can automate portions of this workflow to alleviate caseload and standardize care provided, provided these techniques can address regulatory needs such as safety, accuracy, and transparency to improve clinical translatability. The aim of this research was to develop machine learning models to predict oesophageal cancer MDT treatment decisions and do so in an explainable fashion. METHODS: Historic MDT decisions from University Hospitals Southampton (UHS) trained established ML algorithms (Multinomial Logistic regression (MLR), Random Forests (RF), Extreme Gradient Boosting (XGB), Decision Tree (DT), and Random Survival Forests (RSF)) to perform treatment classification and prognostication tasks. Classification models (MLR, RF, XGB, DT) predicted specific curative and palliative treatment plans, while palliative patients also had their estimated survival predicted when associated with a specific treatment. Classification models were assessed primarily on Area Under the Curve (AUC), while survival forecasts were assessed primarily by calibration curve. All UHS models were externally validated using data from Oxford University Hospitals (OUH). To integrate responsible innovation, transparency and explainability within this research, select eXplainable AI techniques (variable importance and partial dependence analyses) were also employed to examine how individual variables influenced predictions. The final user interface for interacting with the models was also guided using Responsible Research and Innovation (RRI) principles. RESULTS: UHS models were trained from a total cohort of 953 cases and validated on 978 OUH cases. Model performance generalised regardless of algorithms and between treatment centres. XGB performed best for the primary classification model (mean AUC 0.909±0.044) whereas MLR demonstrated the best generalisability between centres (0.894±0.056). XGB performed best on palliative treatment classification both locally and externally (0.815±0.081, 0.742±0.064 respectively). The palliative survival model calibrated best in the first 12 months post-diagnosis for both cohorts. During this work, XAI techniques identified age as a significant influence on treatment allocation. Partial dependence analysis narrowed down the precise age at which probabilities for treatments shifted, as approximately 77 years. CONCLUSION: Within this thesis I have shown that ML techniques can successfully model oesophageal cancer MDT treatment decisions and in a select subset: survival. These models support decision-making early within the MDT pathway. High-performing AI-based decision-support for the OC MDT is technically possible when combined with eXplainable AI methods to provide transparency for regulators as well as driving insight into potential biases within MDT-based decision-making. While future models might benefit from integration of raw imaging data and novel molecular markers, ML can synergize with current MDT frameworks. In future, this can evolve to prioritizing caseload, accelerating decision-making, and providing data-driven support for counselling patients in clinic when discussing treatment plans.