Performance of an artificial intelligence/machine learning model designed to identify hospitalized patients with cancer who could benefit from timely specialized palliative care delivery.

1559 Background: Timely access to specialty Palliative Care (SPC) services provides significant benefits to hospitalized patients with cancer, including improvements in quality of life. However, many patients with cancer who could benefit from SPC suffer from lack of timely referral or do not receive SPC services. A previously published trial of an artificial intelligence/machine learning (AI/ML) model to predict need for SPC at Mayo Clinic increased timely SPC consultation of hospitalized patients overall. We sought to review the performance of this algorithm in patients with cancer across an expanded hospitalized population. Methods: The study population consisted of all patients admitted into a Mayo Clinic hospital in Minnesota, Wisconsin, Arizona and Florida between January 2020 and September 2023. Due to the size of the cohort a case-control sample of three controls for every case was created. Patients were considered to be a patient with cancer if they had at least one billing diagnosis up to 1 year prior to their index hospitalization from any of the 5 HCC categories denoting cancer: Metastatic Cancer and Acute Leukemia (HCC 8); Lung and Other Severe Cancers (HCC 9); Lymphoma and Other Cancers (HCC 10); Colorectal, Bladder, and Other Cancers (HCC 11); and Breast, Prostate, and Other Cancers and Tumors (HCC 12). The training data set consisted of 107,076 patient encounters with a total of 8,355,090 time periods of constant risk. An AI/ML model using gradient boosting methods which contained 269 variables (both static and time-varying) of various classes with SPC consultation treated as a time-to-event outcome was trained. Results: Due to the longitudinal nature of the prediction, performance was assessed using the max score AUC (using the max score a patient received during a given encounter prior to an event or discharge) to produce the AUC. The model had an overall AUC of 0.932 (0.929, 0.935 – 95% CI). We saw a decrease in performance of the AI/ML model in the Oncology population with an overall performance of 0.818 (0.806, 0.83). Some of the most influential variables were previous SPC visit, age, metastatic disease, acute leukemia diagnosis, and pain scores. Conclusions: An AI/ML model can effectively predict the need for an inpatient PC consult in a population of hospitalized patients with cancer. Table: see text