Quality Improvement Demands Quality Data

In this issue of Anesthesia & Analgesia, Liau et al.1 provide a detailed description of how the Anesthesia Quality Institute (AQI) enrolls practices, manages the data collection process for the National Anesthesia Clinical Outcomes Registry (NACOR), ensures data security, meets current requirements of the Health Insurance Portability and Accountability Act of 1996,2 provides feedback to individual practices, and makes data available to researchers. The authors explain that the mission of the AQI is “to promote quality improvement in anesthesiology,” with its primary purpose related to “local practice quality improvement.”1 Benchmarking at the national level is provided for selected metrics. Data files are available to investigators in participating practices for specified projects, after local IRB approval and authorization through processes established by the AQI Data Use Committee. The authors indicate that use of the NACOR database for research purposes is a secondary objective, subject to a broad set of well-described limitations and caveats.1 These issues likely account for the fact that only 4 research studies have been published in peer-reviewed journals as of April 2015.3–6 When the AQI was chartered by the American Society of Anesthesiologists (ASA) in 2008, a major strategic decision was made to focus on the collection of data that were “readily available in digital form,” rather than specifying required data elements necessary to achieve specified, defined purposes.1 The latter approach is more typical of other quality improvement medical registries (e.g., the American College of Surgeons National Surgical Quality Improvement Program®).7 The “hands-off”1 AQI methodology has resulted in a low barrier to participation (i.e., >283 practices contributing to approximately 30,000,000 anesthesia records as of August 2015) and minimal effort and cost to participants (e.g., free to ASA members or $1000 per participant, otherwise; data transfer automated once configured; and no need for practices to hire staff to manage the process).1 However, there are important consequences of the AQI approach that currently limit the utility of the data being collected, both for individual practices and for health services and outcomes researchers. The authors report that 32.5% of practices are reporting outcomes,1 but based on the examples provided in the manuscript and the outcome templates currently on the AQI Web site,8 these are nearly all adverse events occurring during the immediate perioperative period. Collection of postdischarge morbidity and mortality (e.g., at 30 days or longer) cannot be accomplished within the AQI data model, because there is not a systematic process that can identify automatically such events using a de-identified data set. For example, consider a patient who undergoes a procedure at one facility and is discharged without incident on postoperative day 1. The patient sustains a complication and is admitted to a different hospital on postoperative day 3. This generally would not be identified. Even more concerning, if the patient dies on postoperative day 23, this would be missed. Within NACOR, preoperative medical conditions are not being collected systematically, necessary for appropriate propensity matching of morbidity and mortality outcomes. For long-term postoperative outcomes to be studied, individual follow-up of every patient or randomly selected patients would need to be conducted at the local level (e.g., as done using the National Surgical Quality Improvement Program), a process inconsistent with the low-cost, low-effort approach embraced by the AQI and its participating facilities. Such follow-up is not a practical approach for many organizations. For perspective, see the recent article from the Cleveland Clinic describing increased mortality after intraoperative use of etomidate.9 Although long-term postoperative outcomes cannot be assessed using NACOR, it would be a realizable objective to measure immediate perioperative adverse events (i.e., while the patient is still in the hospital). This could be accomplished if the AQI were to require, as a condition of participation, that such events are reported; otherwise, there will be no quality component related to patient care. Many practices likely will need to implement such a process to capture adverse events and can refer to the work of Vigoda et al.10 for guidance. From the perspective of individual practice feedback, the authors report that only 10% of participating practices are providing detailed data from their anesthesia information management systems; 90% just provide administrative data (e.g., from the practice’s professional fee billing system). This means that many practices do not report operating room (OR) entry and exit times, rather anesthesia billing start and end times. In addition, administrative data sets often lack the physical room where cases were performed. The absence of these data precludes the valid determination of most OR management metrics (e.g., turnover times,11–13 tardiness of first case starts,14–16 case durations,17–19 concurrency,20,21 utilization,22,23 and OR efficiency24–26). By definition, billing data only include performed procedures, not scheduled procedures, because the invoice is based on what was actually done. If scheduled procedures are not provided, this precludes valid comparison and prediction of case durations (e.g., for evaluating the accuracy of case scheduling27–29 and potential reductions in patient fasting30 and arrival times31). This is notwithstanding the fact that when multiple procedures are performed, only the procedure that crosswalks to the ASA code with the largest number of base units may be reported. Furthermore, according to the authors, some practices (15%) are not even reporting the Current Procedural Terminology® surgical codes, just anesthesia Current Procedural Terminology codes, which provide insufficient granularity for OR management purposes. There also is substantial ambiguity (25% of cases as of June 15, 2015)32 for practices providing services in multiple locations as to the type of facility (e.g., freestanding surgical center, hospital, and physician office) where individual cases were performed, preventing comparison of complication rates among various facility types. To correct this, the AQI should strongly encourage, if not require, practices to report for all cases the facility type, OR enter and exit times, scheduled procedures, and a de-identified actual OR location. Given that the primary objective of the AQI is “local practice quality improvement,” the organization should influence its participants to provide what is needed to accomplish this laudable goal. Because presently there are potentially insurmountable limitations on tracking patient outcomes beyond the immediate postoperative period, it is unreasonable to expect the AQI to be able to provide information on important postoperative outcomes potentially influenced by anesthesia care (e.g., 30-day mortality, persistent cognitive decline, late cancer recurrence, or delayed surgical wound infection). Realistically, the AQI should put this option aside. Perioperative adverse events reasonably could be assessed, but first, a much higher percentage of hospitals need to report adverse events. This might be feasible within a few years. The greatest relatively short-term value to individual practices that are contributing data to NACOR is in the area of improving OR management.33 However, to realize this benefit, a few additional data elements need to be provided for each case (i.e., facility type, OR enter and exit times, scheduled procedure(s), and de-identified OR location) and the NACOR database schema modified accordingly. The addition of these data may enhance the ability of investigators to use NACOR data for research purposes. Although the infrastructure created by the AQI to collect, protect, and manage massive amounts of data is excellent, it is time to address current lapses in the data that are being provided by participating practices. Quality Improvement demands Quality Data. DISCLOSURES Name: Richard H. Epstein, MD. Contribution: This author wrote the manuscript. Attestation: Richard H. Epstein approved the final manuscript. This manuscript was handled by: Franklin Dexter, PhD, MD.