Development and validation of a model predicting mild stroke severity on admission using electronic health record data

OBJECTIVE: Initial stroke severity is a potent modifier of stroke outcomes but this information is difficult to obtain from electronic health record (EHR) data. This limits the ability to risk-adjust for evaluations of stroke care and outcomes at a population level. The purpose of this analysis was to develop and validate a predictive model of initial stroke severity using EHR data elements. METHODS: This observational cohort included individuals admitted to a US Department of Veterans Affairs hospital with an ischemic stroke. We extracted 65 independent predictors from the EHR. The primary analysis modeled mild (NIHSS score 0-3) versus moderate/severe stroke (NIHSS score ≥4) using multiple logistic regression. Model validation included: (1) splitting the cohort into derivation (65%) and validation (35%) samples and (2) evaluating how the predicted stroke severity performed in regard to 30-day mortality risk stratification. RESULTS: The sample comprised 15,346 individuals with ischemic stroke (n = 10,000 derivation; n = 5,346 validation). The final model included 15 variables and correctly classified 70.4% derivation sample patients and 69.4% validation sample patients. The areas under the curve (AUC) were 0.76 (derivation) and 0.76 (validation). In the validation sample, the model performed similarly to the observed NIHSS in terms of the association with 30-day mortality (AUC: 0.72 observed NIHSS, 0.70 predicted NIHSS). CONCLUSIONS: EHR data can be used to construct a surrogate measure of initial stroke severity. Further research is needed to better differentiate moderate and severe strokes, enhance stroke severity classification, and how to incorporate these measures in evaluations of stroke care and outcomes.