Quality Improvement Interventions Associated With Improved Postoperative Nausea and Vomiting: Separating the Signal From the Noise

See Article, p All models are wrong, but some models are useful. —George E. P. Box (1979) In this issue of Anesthesia & Analgesia, Gabel et al1 report on the success of their quality improvement initiative to improve postoperative nausea and vomiting. We address 3 aspects of the work in this editorial. We first comment on the utilized enhanced recovery after surgery strategy in the study by Gabel et al,1 and how interventions to further reduce postoperative nausea and vomiting might be enhanced going forward. Second, we discuss the vital role of the anesthesiologist in continuous quality improvement initiatives and the choice of Gabel et al1 to study multiple, overlapping interventions. Finally, we discuss salient features of segmented regression for an interrupted time series, the statistical method implemented by Gabel et al1 to compare their pre- and postintervention periods. POSTOPERATIVE NAUSEA AND VOMITING AND ENHANCED RECOVERY AFTER SURGERY Evidence-based antiemetic prophylactic administration strategies have been shown to reduce the incidence of postoperative nausea and vomiting after surgery.2–4 With the increasing adoption of the enhanced recovery after surgery strategy, postoperative nausea and vomiting reduction protocol is an integral part of the larger enhanced recovery protocol in a variety of surgical procedures.5 However, the challenge has been to improve compliance specifically the risk factor score-based prophylactic antiemetic administration protocol. In this cost-effectiveness study, Gabel et al1 report a digital quality improvement approach using a clinical decision support tool integrated with an electronic medical record as well as a modified Apfel Risk Assessment Scoring system.6 This system includes female sex, nonsmoking status, and previous history of postoperative nausea and vomiting or motion sickness, but not perioperative opioid usage. Instead, it includes a number of surgical procedures that have traditionally been associated with a high risk for postoperative nausea and vomiting.7 The authors also added a “very high-risk” group, defined as any patients who were verified to have received appropriate antiemetic prophylaxis during previous procedures but still experienced postoperative nausea and vomiting. Several findings are of interest. First, it should be noted that the authors used the administration of a rescue antiemetic in the postanesthesia care unit (PACU) as a surrogate for a postoperative nausea and vomiting episode. This may lead to inaccurate reporting because patients who experience mild to moderate nausea and even vomiting may not receive an antiemetic due to patient preference and other factors. In addition, the observation of postoperative nausea and vomiting symptoms limited to PACU is likely to underestimate the true incidence within 24 hours and most likely by 10%–15% as demonstrated by most postoperative nausea and vomiting studies.8 It is also unclear if the antiemetic drug administered in the PACU was for postoperative nausea and vomiting indication because some “traditional” antiemetics may be administered for other indications (eg, diphenhydramine for treatment of pruritus and haloperidol for confusion). Second, the administration of prophylactic antiemetics resulted in a greater reduction in the incidence of postoperative nausea and vomiting in the high-risk patients with ≥3 risk factors (6% reduction from 29.3% to 23.5%) compared to lower-risk patients with 1 of 2 risk factors (2.2% reduction from 19.1% to 16.9%). Hence, about 1 in 4 patients in the high-risk group still developed postoperative nausea and vomiting despite appropriate antiemetic prophylaxis. Third, the compliance rate after implementation of the clinical decision support is around 85%. The automatic weekly e-mails sent to the providers giving detailed information on the patients and their outcome data did not further improve compliance rate. Can we improve on these results? Possibly. Given that the compliance rate did not change significantly after initiation of the weekly e-mail reports, a more effective method may be using a more active clinical decision support system within the electronic medical record. For example, the administration of appropriate prophylactic antiemetics may be included as a mandatory step in the electronic medical record unless rejected by the providers for specific contraindications. This has to be balanced with the downside of this approach, specifically “user fatigue.” Even with near-perfect compliance, it is probably unlikely that one would see a near-zero incidence of postoperative nausea and vomiting unless we have a better understanding of the underlying cause and mechanism of postoperative nausea and vomiting or more efficacious antiemetics. Current antiemetics, when administered alone, are only capable of reducing the risk by about 25%. Even with the administration of 3–4 antiemetics in high-risk patients, it is difficult to reduce the incidence below 10%–15%.9 CONTINUOUS QUALITY IMPROVEMENT Significant changes in population demographics and health policy are mandating innovative yet sustainable value-based models of health care delivery including for surgical patient care.10,11 Anesthesiologists are well positioned to assume a broader yet continued highly collaborative role in the perioperative care of surgical patients.12 Anesthesiologists likewise have considerable experience and demonstrated expertise in the fields of performance improvement and continuous quality improvement.13 They can thus serve as very active leaders in this new perioperative care environment14,15including in the delivery and demonstration of successful perioperative population health management.16–18 For example, the Anesthesia Quality Institute implemented the National Anesthesia Clinical Outcomes Registry to automatically capture robust electronic data specific to anesthesia cases. These outcomes and health services data have been rigorously extracted simultaneously from billing systems, quality management systems, hospital electronic health care records, and anesthesia information management systems.13,18 Against this backdrop, Gabel et al1 examined whether an electronic medical record–based clinical decision support approach—that sequentially incorporated 3 elements: a new postoperative nausea and vomiting pathway, a clinician education initiative, and a personalized feedback reporting system—decreased the incidence of postoperative nausea and vomiting. Their clinical decision support tools included additional postoperative nausea and vomiting–specific questions on the electronic preoperative evaluation and risk stratification form, creation of a real-time intraoperative pathway compliance indicator, initiation of preoperative postoperative nausea and vomiting risk alerts, and individualized emailed reports sent weekly to clinical providers. While certainly innovative, the sequential 3-stage continuous quality improvement approach to preventing postoperative nausea and vomiting applied by Gabel et al1 does raise several key questions. While ostensibly logical, was it a fundamentally valid approach to study the effects of all 3 interrelated continuous quality improvement elements at the same time? What inferences and conclusions can be made about the relative contribution of each of the 3 elements? Which of the 3 elements were responsible for the overall observed positive effect of the incidence of postoperative nausea and vomiting? For readers considering implementing ≥1 of the 3 elements, which would have the greatest return on the investment or would all 3 elements be required to realize any observed patient benefit? SEGMENTED REGRESSION Gabel et al1 sought to assess the effect of their 3-pronged intervention on the proportion of patients having postoperative nausea and vomiting. They compared pre- and postintervention periods, each 1 year in length and completely separated by time, on the outcomes of interest in a before-and-after design. Such quality improvement initiatives are fraught with inherent biases19 which can invalidate the results unless careful design and analyses are implemented.20 Observed “improvements” might be a continuation of the preintervention trend. Hawthorne effect21 may be an issue because there is no placebo effect in the preintervention phase; subjects may respond positively during intervention when given extra attention or monitoring. Regression to the mean22,23 may result if intervention was initiated because of a temporal spike in outcomes that would have resolved without the intervention.20 In attempts to avoid these biases, Gabel et al1 did not simply compare the pre- and postintervention periods on the proportion experiencing postoperative nausea and vomiting. They used segmented regression to compare the periods on the slope of the postoperative nausea and vomiting over time, and to test for a decrease, or improvement, at start of intervention.20,24 Segmented regression showed no change in slope over time but a significant decrease comparing the end of preintervention to start of postintervention (P = .007; Table 2). They further found that the intervention effect was a function of baseline postoperative nausea and vomiting risk with an improvement for patients at high risk of postoperative nausea and vomiting but with no effect for low-risk patients (interaction P < .001). Because the 3 interventions started at different times, Gabel et al1 also conducted a more specific segmented regression to estimate slopes and intercepts corresponding to each intervention. However, because the interventions overlapped in time, deciphering which intervention is responsible for any effect would be challenging. Finally, Gabel et al1 used their segmented regression model to compare the observed postintervention postoperative nausea and vomiting proportion at specific time points to the projected “counterfactual” proportions if the preintervention trend would have instead continued.24 Conducting segmented regression to assess the effect of a new policy or intervention assumes that the data represent an interrupted time series.25,26 However, proper analysis of an interrupted time series requires a sufficient time window both pre- and postintervention (eg, 1 year in each) as well as a sufficient number of time points (eg, 10) in each period. A sufficient time window allows proper assessment of the trends over time including whether the trend is stable and linear or not, and these must be verified before conducting the analysis.20 These conditions do seem to have been met in Gabel et al.1 The chosen segmented regression model should also be appropriate for the outcome variable, such as linear regression for continuous outcomes, and logistic regression for binary. For example, using linear regression on the mean of a binary variable can give incorrect conclusions. Segmented regression can be conducted on data aggregated into time intervals (eg, 1 week or month) or else on the patient level with the latter allowing inclusion of more detail on the outcome and confounding variables, and the former perhaps removing some of the autocorrelation in the time series.20,27 Much to their credit, Gabel et al1 adjusted for confounding in their segmented regression. They did this by first matching pre- and postintervention patients using propensity score matching.28 Confounding variables can also be added directly into the segmented regression. It is key to thoroughly adjust for confounding in a segmented regression analysis even if it is hypothesized that potentially confounding variables might be stable over time.20 Finally, segmented regression is the preferred method for assessing the effect of a quality improvement program in absence of a concurrent control. However, when there is a concurrent cohort which did not receive the intervention, difference-in-difference methods29–32 can be used to compare changes in the experimental cohort to those in the control cohort, generally permitting stronger inference on the treatment effect. In summary, Gabel et al1 present a fairly well-designed and analyzed digital quality improvement initiative to reduce postoperative nausea and vomiting. They demonstrate that persistence in seeking novel ways to improve important clinical outcomes can pay off especially when implementing evidence-based and perhaps multifaceted interventions. Proper design and analysis of such initiatives, such as segmented regression of an interrupted time series or difference-in-difference methods, are essential for their success. DISCLOSURES Name: Edward J. Mascha, PhD. Contribution: This author helped write and revise the manuscript. Name: Tong J. Gan, MD, MBA, FRCA, MHS, LiAc. Contribution: This author helped write and revise the manuscript. Name: Thomas R. Vetter, MD, MPH. Contribution: This author helped write and revise the manuscript. This manuscript was handled by: Jean-Francois Pittet, MD.