Impact of Missing Stroke Severity Data on the Accuracy of Hospital Ischemic Stroke Mortality Profiling

BACKGROUND: The Centers for Medicare and Medicaid Services have proposed 30-day ischemic stroke risk-standardized mortality rates that include adjustment for stroke severity using the National Institute of Health Stroke Scale (NIHSS), which is often undocumented. We used simulations to quantify the effect of missing NIHSS data on the accuracy of hospital-level ischemic stroke risk-standardized mortality rate profiling for 100 hypothetical hospitals with different case volumes. METHODS AND RESULTS: We generated simulated data sets of patients with NIHSS scores and other predictors of 30-day mortality based on empirical analysis of data from 7654 patients with ischemic stroke in the Michigan Stroke Registry. We assigned and rank-ordered a true (known) 30-day mortality rate to each hospital in the simulated data sets of N=100 hospitals with either low (100 patients with stroke), medium (300), or high (500) case volumes. We then estimated and rank-ordered 30-day risk-standardized mortality rates for the N=100 hospitals in each simulated data set using hierarchical logistic regression models. In each data set, we systematically varied the rate of missing NIHSS data and whether missing NIHSS data was independent (missing completely at random) or dependent (missing not at random) on the NIHSS score. With no missing NIHSS data, the Spearman correlation between the true hospital performance rank order assigned during data set generation and the estimated 30-day risk-standardized mortality rate rank order was 0.72, 0.88, and 0.91 in low, medium, and high volume hospitals, respectively. However, this fell to as low as 0.50, 0.71, and 0.79 as missing NIHSS data reached 70%. CONCLUSIONS: Missing NIHSS data had substantial detrimental effects on the accuracy of profiling of ischemic stroke mortality, especially in lower volume hospitals. Even with complete NIHSS documentation, significant limitations in ischemic stroke mortality profiling remain.