Limitations to Health Care Quality Measurement: Assessing Hospital Variation in Risk of Cardiac Events After Noncardiac Surgery

Limited sample size, incomplete measures, and inadequate risk adjustment adversely influence accurate health care quality measurements, surgical quality measurements, and accurate comparisons among hospitals. Since these measures are linked to resources for quality improvement and reimbursement, improving the accuracy of measurement has substantial implications for patients, clinicians, hospital administrators, insurers, and purchasers. The team examined risk-adjusted differences of postoperative cardiac events among 20 geographically dispersed, community-based medical centers within an integrated health care system and compared it with the National Surgical Quality Improvement Program (NSQIP) hospital-specific differences. The exposure included the hospital at which patients received noncardiac surgical care, with stratification of patients by the acuity of surgery (elective vs. urgent/emergent). Among 157,075 surgery patients, the unadjusted risk of cardiac event per 1000 ranged among hospitals from 2.1 to 6.9 for elective surgery and from 10.3 to 44.5 for urgent/emergent surgery. Across the 20 hospitals, hospital rankings estimated in the present analysis differed significantly from ranking reported by NSQIP ( P for difference: elective, P = 0.0001; urgent/emergent, P < 0.0001) with significantly and substantially lower variation after risk adjustment. Current surgical quality measures may not adequately account for limitations of sample size, data capture, adequate risk adjustment, and surgical acuity in a given hospital, particularly for rare outcomes. These differences have implications for quality reporting and may introduce bias into hospital comparisons, particularly for hospitals with incomplete capture of their patients' baseline risk and acuity.