Relevance of the Agency for Healthcare Research and Quality Patient Safety Indicators for Children's Hospitals

OBJECTIVES: Patient safety indicators (PSIs) were developed by the Agency for Healthcare Research and Quality. Our objectives were (1) to apply these algorithms to the National Association of Children's Hospitals and Related Institutions (NACHRI) Aggregate Case Mix Comparative Database for 1999-2002, (2) to establish mean rates for each of the PSI events in children's hospitals, (3) to investigate the inadequacies of PSIs in relation to pediatric diagnoses, and (4) to express the data in such a way that children's hospitals could use the PSIs determined to be appropriate for pediatric use for comparison with their own data. In addition, we wanted to use the data to set priorities for ongoing clinical investigations and to propose interventions if the indicators demonstrated preventable errors. METHODS: The Agency for Healthcare Research and Quality PSI algorithms (version 2.1, revision 1) were applied to children's hospital administrative data (1.92 million discharges) from the NACHRI Aggregate Case Mix Comparative Database for 1999-2002. Rates were measured for the following events: complications of anesthesia, death in low-mortality diagnosis-related groups (DRGs), decubitus ulcer, failure to rescue (ie, death resulting from a complication, rather than the primary diagnosis), foreign body left in during a procedure, iatrogenic pneumothorax, infection attributable to medical care (ie, infections related to surgery or device placement), postoperative hemorrhage or hematoma, postoperative pulmonary embolism or venous thrombosis, postoperative wound dehiscence, and accidental puncture/laceration. RESULTS: Across the 4 years of data, the mean risk-adjusted rates of PSI events ranged from 0.01% (0.1 event per 1000 discharges) for a foreign body left in during a procedure to 14.0% (140 events per 1000 discharges) for failure to rescue. Review of International Classification of Diseases, Ninth Revision, Clinical Modification codes associated with each PSI category showed that the failure to rescue and death in low-mortality DRG indicators involved very complex cases and did not predict preventable events in the majority of cases. The PSI for infection attributable to medical care appeared to be accurate the majority of the time. Incident risk-adjusted rates of infections attributable to medical care averaged 0.35% (3.5 events per 1000 discharges) and varied up to fivefold from the lowest rate to the highest rate. The highest rates were up to 1.8 times the average. CONCLUSIONS: PSIs derived from administrative data are indicators of patient safety concerns and can be relevant as screening tools for children's hospitals; however, cases identified by these indicators do not always represent preventable events. Some, such as a foreign body left in during a procedure, iatrogenic pneumothorax, infection attributable to medical care, decubitus ulcer, and venous thrombosis, seem to be appropriate for pediatric care and may be directly amenable to system changes. Evidence-based practices regarding those particular indicators that have been reported in the adult literature need to be investigated in the pediatric population. In their present form, 2 of the indicators, namely, failure to rescue and death in low-mortality DRGs, are inaccurate for the pediatric population, do not represent preventable errors in the majority of pediatric cases, and should not be used to estimate quality of care or preventable deaths in children's hospitals. The PSIs can assist institutions in prioritizing chart review-based investigations; if clusters of validated events emerge in reviews, then improvement activities can be initiated. Large aggregate databases, such as the NACHRI Case Mix Database, can help establish mean rates of potential pediatric events, giving children's hospitals a context within which to examine their own data.