Positive Predictive Value of the AHRQ Patient Safety Indicator “Postoperative Sepsis”: Implications for Practice and Policy

BACKGROUND: Patient Safety Indicator (PSI) 13, or "Postoperative Sepsis," of the Agency for Healthcare Quality and Research (AHRQ), was recently adopted as part of a composite measure of patient safety by the Centers for Medicare and Medicaid Services (CMS). We sought to examine its positive predictive value (PPV) by determining how well it identifies true cases of postoperative sepsis. STUDY DESIGN: Two retrospective cross-sectional studies of hospitalization records that met PSI 13 criteria were conducted, one within the Veterans Administration (VA) Hospitals from fiscal years (FY) 2003 to 2007, and one within community hospitals between October 1, 2005 and March 31, 2007. Trained abstractors reviewed medical records from each database using standardized abstraction instruments. We determined the PPV of the indicator and performed descriptive analyses of cases. RESULTS: Of 112 cases flagged and reviewed within the VA system, 59 were true events of postoperative sepsis, yielding a PPV of 53% (95% CI 42% to 64%). Within the community hospital sector, of 164 flagged and reviewed cases, 67 were true cases of postoperative sepsis, yielding a PPV of 41% (95% CI 28% to 54%). False positives were due to infections that were present on admission, urgent or emergent cases, no clinical diagnosis of sepsis, or other coding limitations such as nonspecific shock in postoperative patients. CONCLUSIONS: PSI 13 has relatively poor predictive ability to identify true cases of postoperative sepsis in both the VA and nonfederal sectors. The lack of information on diagnosis timing, confusion about the definition of elective admission, and coding limitations were the major reasons for false positives. As it currently stands, the use of PSI 13 as a stand-alone measure for hospital reporting appears premature.