Validating risk-adjusted surgical outcomes: chart review of process of care

OBJECTIVE: The primary purpose of this study was to validate risk-adjusted surgical outcomes as indicators of the quality of surgical care at US Department of Veterans Affairs (VA) hospitals. The secondary purpose was to validate the risk-adjustment models for screening cases for quality review. DESIGN: We compared quality of care, determined by structured implicit chart review, for patients from hospitals with higher and lower than expected operative mortality and morbidity (hospital-level tests) and between patients with high and low predicted risk of mortality and morbidity who died or developed complications (patient-level tests). SUBJECTS: 739 general, peripheral vascular and orthopedic surgery cases sampled from the 44 VA hospitals participating in the National VA Surgical Risk Study. MAIN OUTCOME MEASURES: A global rating of quality of care based on chart review. RESULTS: Ratings of overall quality of care did not differ significantly between patients from hospitals with higher and lower than expected mortality and morbidity. On some of the secondary measures, patient care was rated higher for hospitals with lower than expected operative mortality. At the patient level of analysis, those who died or developed complications and had a high predicted risk of mortality or morbidity were rated higher on quality of care than those with a low predicted risk of adverse outcome. CONCLUSIONS: The absence of a relationship between most of our measures of process of care and risk-adjusted outcomes may be due to an insensitivity of chart reviews to hospital-level differences in quality of care. Site visits to National VA Surgical Risk Study hospitals with high and low risk-adjusted mortality and morbidity have detected differences on a number of dimensions of quality. The patient-level findings suggest that the risk-adjustment models are useful for screening adverse outcome cases for quality of care review.