Key result
Applying a predictive model for 30-day death or readmission to all hospitalizations per patient rather than one decreased the c-statistic from 0.768 to 0.730.
Why the study?
Does applying a readmission prediction model to all hospitalizations rather than one per patient decrease model performance?
Observational (n=499,996)
Does applying a readmission prediction model to all hospitalizations rather than one per patient decrease model performance?
Absolute Event Rate: 1.305% vs 0.998%
Predictive models for readmission or death lose accuracy when applied to multiple hospitalizations per patient, particularly in high-utilization patients.
Performance shifts when models include multiple hospitalizations per patient; leaves open optimal dataset construction for readmission prediction.
BACKGROUND: To avoid biased estimates of standard errors in regression models, statisticians commonly limit the analytical dataset to one observation per patient. OBJECTIVE: Measure and explain changes in model performance when a model predicting 30-day risk of death or urgent readmission (derived on a dataset having one hospitalization per patient) was applied to all hospitalizations for study patients. METHODS: Using administrative data from Ontario, we identified all hospitalizations of 499,996 patients between 2004 and 2009. We calculated the expected risk for 30-day death or urgent readmission using a validated model. The observed-to-expected ratio was determined after categorizing patients into quintiles of rates for hospitalization, emergent hospitalizations, hospital day and total diagnostic risk score. RESULTS: Study patients had a total of 858,410 hospitalizations. Compared with a dataset having one hospitalization per patient, model performance declined significantly when applied to all hospitalizations [c-statistic decreased from 0.768 to 0.730; the observed-to-expected ratio increased from 0.998 (95% confidence interval 0.977-0.999) to 1.305 (1.297-1.313)]. Model deterioration was most pronounced in patients with higher hospital utilization, with the observed-to-expected ratio increasing to 1.67 in the highest quintile of emergent hospitalization rates. CONCLUSIONS: The accuracy of predicting 30-day death or urgent readmission decreased significantly when the unit of analysis changed from the patient to the hospitalization. Patients with heavy hospital utilization likely have characteristics, not adequately captured in the model, that increase the risk of death or urgent readmission after discharge from hospital. Adequately capturing the characteristics of such high-end hospital users may improve readmission models.
No takes yet. Share an insight, caveat, or question.
Walraven et al. (2012) conducted an observational in Hospitalization (n=499,996). Applying predictive model to all hospitalizations per patient vs. Applying predictive model to one hospitalization per patient was evaluated on Model performance (observed-to-expected ratio) for 30-day death or urgent readmission (95% CI 1.297-1.313). Applying a predictive model for 30-day death or readmission to all hospitalizations per patient rather than one decreased the c-statistic from 0.768 to 0.730.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: