Comparison of ICD-9–based, retrospective, and prospective assessments of perioperative complications: assessment of accuracy in reporting

OBJECT: large studies of ICD-9-based complication and hospital-acquired condition (HAC) chart reviews have not been validated through a comparison with prospective assessments of perioperative adverse event occurrence. Retrospective chart review, while generally assumed to underreport complication occurrence, has not been subjected to prospective study. It is unclear whether ICD-9-based population studies are more accurate than retrospective reviews or are perhaps equally susceptible to bias. To determine the validity of an ICD-9-based assessment of perioperative complications, the authors compared a prospective independent evaluation of such complications with ICD-9-based HAC data in a cohort of patients who underwent spine surgery. For further comparison, a separate retrospective review of the same cohort of patients was completed as well. METHODS: a prospective assessment of complications in spine surgery over a 6-month period (May to December 2008) was completed using an independent auditor and a validated definition of perioperative complications. The auditor maintained a prospective database, which included complications occurring in the initial 30 days after surgery. All medical adverse events were included in the assessment. All patients undergoing spine surgery during the study period were eligible for inclusion; the only exclusionary criterion used was the availability of the auditor for patient assessment. From the overall patient database, 100 patients were randomly extracted for further review; in these patients ICD-9-based HAC data were obtained from coder data. Separately, a retrospective assessment of complication incidence was completed using chart and electronic medical record review. The same definition of perioperative adverse events and the inclusion of medical adverse events were applied in the prospective, ICD-9-based, and retrospective assessments. RESULTS: ninety-two patients had adequate records for the ICD-9 assessment, whereas 98 patients had adequate chart information for retrospective review. The overall complication incidence among the groups was similar (major complications: ICD-9 17.4%, retrospective 19.4%, and prospective 22.4%; minor complications: ICD-9 43.8%, retrospective 31.6%, and prospective 42.9%). However, the ICD-9-based assessment included many minor medical events not deemed complications by the auditor. Rates of specific complications were consistently underreported in both the ICD-9 and the retrospective assessments. The ICD-9 assessment underreported infection, the need for reoperation, deep wound infection, deep venous thrombosis, and new neurological deficits (p = 0.003, p < 0.0001, p < 0.0001, p = 0.0025, and p = 0.04, respectively). The retrospective review underestimated incidences of infection, the need for revision, and deep wound infection (p < 0.0001 for each). Only in the capture of new cardiac events was ICD-9-based reporting more accurate than prospective data accrual (p = 0.04). The most sensitive measure for the appreciation of complication occurrence was the prospective review, followed by the ICD-9-based assessment (p = 0.05). CONCLUSIONS: an ICD-9-based coding of perioperative adverse events and major complications in a cohort of spine surgery patients revealed an overall complication incidence similar to that in a prospectively executed measure. In contrast, a retrospective review underestimated complication incidence. The ICD-9-based review captured many medical events of limited clinical import, inflating the overall incidence of adverse events demonstrated by this approach. In multiple categories of major, clinically significant perioperative complications, ICD-9-based and retrospective assessments significantly underestimated complication incidence. These findings illustrate a significant potential weakness and source of inaccuracy in the use of population-based ICD-9 and retrospective complication recording.