Independent validation of 30-day emergency department visit after initial chemotherapy A-PROACCT model.

e23243 Background: This study assessed model performance and, more importantly, the clinical usefulness of the PROACCT model created by Grant et al. (JAMA, 2019), refined as A-PROACCT by Stein et al. (JOP, 2023) with data from Orlando Health. The predictive performance of models anticipating acute care events within thirty days (ACE30) of initial chemotherapy administration (ICA) were evaluated. Methods: Stein et al. models classified high-risk patients as those whose model output risk score was at or above the 90 th percentile. They studied 5153 ICAs between July and November of 2021, of which 559 (10.8%) had an ACE30. Their first model used drug category, age group, ED visits and hospitalizations in the prior year, and cancer type, and had sensitivity (Sen), specificity, (Spec), C-Statistic (C-Stat), and positive predictive value (PPV) of 0.28, 0.93, 0.79 and 0.18 respectively. Their second model added insurance category, number of anti-cancer agents, race, and BMI category and reported Sen, Spec, C-Stat, and PPV of 0.30, 0.94, 0.80 and 0.23 respectively. Of the high-risk patients, 33.6% and 34.1% had an ACE30 respectively. Of the low-risk patients, 8.3% and 8.1% had an ACE30 respectively. Our data were propagated through the two models. Results: Between February of 2012 and April 2021, Orlando Health had 15,483 distinct patient visits with ICAs. Overall, 2046 (13.2%) had an ACE30. Model one had Sen, Spec, C-Stat, and PPV of 0.19, 0.92, 0.65 and 0.27 respectively. Model two had Sen, Spec, C-Stat, and PPV of 0.19, 0.91, 0.65 and 0.25 respectively. Both models had an accuracy of 0.82. This demonstrated our data fit the Stein model. Of the 1460 ICAs identified as high-risk by model one, 387 (26.5%) had an ACE30, and of the 14,023 ICAs identified as low-risk, 1659 (11.8%) had an ACE30. Of the 1549 ICAs identified as high-risk by model two, 393 (25.4%) had an ACE30, and of the 13,934 ICAs identified as low-risk, 1653 (11.9%) had an ACE30. The difference of ACE30 rates between high risk and low risk groups were statistically significant (p < 0.0001) in both models. This demonstrated both models were clinically useful in identifying chemotherapy patients at high risk of acute care events. Conclusions: These models allowed for clinically relevant stratification of patients into meaningful high-risk and low-risk groups (26.5% vs 11.8% with our data) for risk of acute care events within 30 days of initial chemotherapy administration. Nevertheless, only 26.5% of these high-risk patients actually had an acute care event within thirty days of initial chemotherapy administration. Further research is needed to improve the precision of identifying high-risk patients, by potentially adding more clinical variables, using alternative sampling methods, and employing nonlinear and more complex machine learning methodologies.