July 26, 2011
View Full PaperThe Need for Multiple Measures of Hospital Quality
Key Result
Agreement between different methods of ranking hospital quality of care (process measures vs 30-day mortality or readmission) was poor (weighted κ ≤ 0.15; Spearman correlation ≤ 0.21).
Study Design
Type
Observational (n=19,483)
Multicenter
Yes
Structured PICO
P
Population
19,483 heart failure patients aged ≥ 65 years from 153 hospitals in the Get With the Guidelines-Heart Failure registry linked to Medicare claims.
E
Exposure
Ranking hospitals by composite adherence scores for 5 heart failure process measures, composite adherence scores for emerging quality measures, risk-adjusted 30-day death after admission, and risk-adjusted 30-day readmission after discharge.
O
Outcome
Correlation and agreement between different methods of ranking hospital-based quality of care (weighted κ, Spearman correlation, shift in ranks)
Agreement between different methods of ranking hospital-based quality of care for heart failure is poor, suggesting that profiling quality requires multidimensional ranking methods.
Main Result
Effect estimate: weighted κ ≤ 0.15
Abstract
BACKGROUND: Process and outcome measures are often used to quantify quality of care in hospitals. Whether these quality measures correlate with one another and the degree to which hospital provider rankings shift on the basis of the performance metric is uncertain. METHODS AND RESULTS: Heart failure patients ≥ 65 years of age hospitalized in the Get With the Guidelines-Heart Failure registry of the American Heart Association were linked to Medicare claims from 2005 to 2006. Hospitals were ranked by (1) composite adherence scores for 5 heart failure process measures, (2) composite adherence scores for emerging quality measures, (3) risk-adjusted 30-day death after admission, and (4) risk-adjusted 30-day readmission after discharge. Hierarchical models using shrinkage estimates were performed to adjust for case mix and hospital volume. There were 19 483 patients hospitalized from 2005 to 2006 from 153 hospitals. The overall median composite adherence rate to heart process measures was 85.8% (25th, 75th percentiles 77.5, 91.4). Median 30-day risk-adjusted mortality was 9.0% (7.9, 10.4). Median risk-adjusted 30-day readmission was 22.9% (22.1, 23.5). The weighted κ for remaining within the top 20th percentile or bottom 20th percentile was ≤ 0.15 and the Spearman correlation overall was ≤ 0.21 between the different measures of quality of care. The average shift in ranks was 33 positions (13, 68) when criteria were changed from 30-day mortality to readmission and 51 positions (22, 76) when ranking metric changed from 30-day mortality to composite process adherence. CONCLUSIONS: Agreement between different methods of ranking hospital-based quality of care and 30-day mortality or readmission rankings was poor. Profiling quality of care will require multidimensional ranking methods and/or additional measures.