Can Administrative Data Be Used to Compare Postoperative Complication Rates Across Hospitals?

BACKGROUND: Several quality assessment systems use administrative data to identify postoperative complications, with uncertain validity. OBJECTIVES: To determine how accurately postoperative complications are reported in administrative data, whether accuracy varies systematically across hospitals, and whether serious complications are more consistently reported. DESIGN: Retrospective cohort. SUBJECTS: Nine hundred ninety-one randomly sampled adults who underwent elective lumbar diskectomies at 30 nonfederal acute care hospitals in California in 1990 to 1991. Hospitals with especially low or high risk-adjusted complication rates, and patients who experienced complications, were over sampled. MEASURES: Postoperative complications were specified by reviewing medical literature and consulting clinical experts; each complication was mapped to ICD-9-CM. Hospital-reported complications were compared with our independent recoding of the same records. RESULTS: The weighted sensitivity, specificity, and positive and negative predictive values for reported complications were 35%, 98%, 82%, and 84%, respectively. The weighted sensitivity was 30% for serious, 40% for minor, and 10% for questionable complications. It varied from 21% among hospitals with fewer complications than expected to 45% among hospitals with more complications than expected. Only reoperation, bacteremia/sepsis, postoperative infection, and deep vein thrombosis were reported with at least 60% sensitivity. Half of the difference in risk-adjusted complication rates between low and high outlier hospitals was attributable to reporting variation. CONCLUSIONS: ICD-9-CM complications were underreported among diskectomy patients, especially at hospitals with low risk-adjusted complication rates. The validity of using coded complications to compare provider performance is questionable, even with careful efforts to identify serious events, although these results must be confirmed using more recent data.