Evaluation of Acute Physiology and Chronic Health Evaluation III predictions of hospital mortality in an independent database

OBJECTIVE: To assess the accuracy and validity of Acute Physiology and Chronic Health Evaluation (APACHE) III hospital mortality predictions in an independent sample of U.S. intensive care unit (ICU) admissions. DESIGN: Nonrandomized, observational, cohort study. SETTING: Two hundred eighty-five ICUs in 161 U.S. hospitals, including 65 members of the Council of Teaching Hospitals and 64 nonteaching hospitals. PATIENTS: A consecutive sample of 37,668 ICU admissions during 1993 to 1996; including 25,448 admissions at hospitals with >400 beds and 1,074 admissions at hospitals with <200 beds. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We used demographic, clinical, and physiologic information recorded during ICU day 1 and the APACHE III equation to predict the probability of hospital mortality for each patient. We compared observed and predicted mortality for all admissions and across patient subgroups and assessed predictive accuracy using tests of discrimination and calibration. Aggregate hospital death rate was 12.35% and predicted hospital death rate was 12.27% (p =.541). The model discriminated between survivors and nonsurvivors well (area under receiver operating curve = 0.89). A calibration curve showed that the observed number of hospital deaths was close to the number of deaths predicted by the model, but when tested across deciles of risk, goodness-of-fit (Hosmer-Lemeshow statistic, chi-square = 48.71, 8 degrees of freedom, p< .0001) was not perfect. Observed and predicted hospital mortality rates were not significantly (p < .01) different for 55 (84.6%) of APACHE III's 65 specific ICU admission diagnoses and for 11 (84.6%) of the 13 residual organ system-related categories. The most frequent diagnoses with significant (p < .01) differences between observed and predicted hospital mortality rates included acute myocardial infarction, drug overdose, nonoperative head trauma, and nonoperative multiple trauma. CONCLUSIONS: APACHE III accurately predicted aggregate hospital mortality in an independent sample of U.S. ICU admissions. Further improvements in calibration can be achieved by more precise disease labeling, improved acquisition and weighting of neurologic abnormalities, adjustments that reflect changes in treatment outcomes over time, and a larger national database.