To Test or Not to Test … Decision Analysis of Decision Support

In this issue of Academic Emergency Medicine, Troyer and colleagues1 present a provocative decision analysis study exploring the cost and benefits associated with a web-based quantitative pretest probability calculator. This was a secondary analysis of data from their previous multicenter randomized controlled trial that enrolled patients with chest pain and/or dyspnea, with emergency physicians receiving either output of a quantified pretest probability calculator or sham.2 Short-term outcomes of their original work included benefit in reduced radiation test ordering without sacrifices in safety. Their decision analysis work presented in this issue explores the likely longer-term outcomes and costs of such an approach. There are several aspects of this work that deserve comment. First, what is a decision analysis? Readers have seen an increasing number of these studies in this as well as other journals.3-6 On the surface these works may appear to be simple mental-computer explorations. However, decision analysis has an important role when 1) the potential study is impossible to do in a prospective randomized manner and 2) there is uncertainty over the probabilities or values of certain events that may happen over time. In these situations, a computer model may be created that inputs everything, from the probability of having a disease, the probability of certain tests being positive or negative, the potential subsequent benefits or harms in terms of health state, and also the values people may place on being in a certain health states. These studies typically use software to process this information to result in a model, or prediction, of what would be mathematically likely given a large number of patients over time. The question of to test or not to test for pulmonary embolism (PE) or acute coronary syndrome (ACS) could ideally be analyzed by a decision analysis model. We would love to know with certainty what the long-term effects of a false-positive computed tomography (CT) scan for PE or cardiac catheterization may be or the likelihood that a given individual will develop cancer as a result of a single or repeat CT or nuclear medicine cardiac test—but we cannot realistically do a study to determine these questions prospectively. Additionally, we cannot know with certainty the probabilities of outcomes such as contrast-induced nephropathy, cancer, or fatal intracranial hemorrhage attributable to anticoagulation. Estimates in the literature exist, but all have some degree of uncertainty surrounding these probabilities. We do not know about long-term costs and benefits of using computed pretest probabilities for patients with symptoms of possible PE or ACS, so this question is a good candidate for a decision analysis approach. There are several unique things about the decision analysis presented by Troyer et al. First, they use test and outcome data from their previously published randomized controlled trial of a quantified pretest probability decision support tool to inform the model. This allows them to use “real-world” data to enter into the model including rates of radiation exposure and contrast exposure, as well as rates and results of testing. Second, they use a microsimulation approach to track “patients” through the decision analysis model (i.e., cycles that keep track of outcome/disease states such as PE, myocardial infarction, renal failure, and intracranial hemorrhage, as well as exposure values such as number of radiation tests). In this way they take the value of data from their randomized controlled trial to start the computer model, and then they are able to use the computer simulation to forecast decades of health states for these patients to estimate final costs of care and outcomes. In short, the authors, having shown in their previous work that quantified pretest probability decision support reduced short-term radiation while maintaining safety, are now using that data to model what would likely happen to these patients over their remaining life spans. Results from this work suggest that when costs and event probabilities are accounted for, under several important assumptions, there is a lifetime cost savings of approximately $500 for those who undergo quantitative decision analysis. Further, they maintain roughly the same quality of life as those randomized to the sham arm. It is important to note that a fundamental aspect of decision analysis is the perspective taken to interpret the values. One might look at this from the perspective of an individual patient who we can imagine would want to maximize his or her health state, no matter what the cost, or we can examine this form the perspective of society or a health care system interested in maximizing the health state of as many people as possible for the lowest possible cost. On the surface, one can scan the abstract and be critical of the modest per-patient health care savings. However, the authors explore ramifications of this from the larger perspective of nationwide adoption and extrapolate to the entire U.S. population, suggesting potential annual savings of $524 billion from a theoretical avoidance of over 900,000 CT scans. There are important limitations to decision analysis, mostly associated with the assumptions and uncertainty inherent to the model. In this work one can question what the true rate of contrast-induced nephropathy is,7 what the true rate of malignancy attributed to diagnostic imaging is, or how valid utility estimates are for health outcomes such as myocardial infarction, nonfatal intracranial hemorrhage, and renal failure. The authors acknowledge these uncertainties but draw on what their interpretation is of the best estimates in the existing literature, and again, their use of real trial data to start the model at time zero helps reduce guesswork for these starting parameters. The essential question of how to provide the greatest good for the greatest number at the lowest cost continues to be a large part of future health care transformation. Despite EDs being only a small portion of total health care expenditures, decisions made in the ED have important downstream ramifications on cost and health states. The decision to test or not test for a condition is among the most consequential that we make and often hinges on advanced imaging. The Society for Academic Emergency Medicine is at the forefront of this discussion with this year's Academic Emergency Medicine consensus conference, which addresses this, among other issues related to emergency department diagnostic imaging. In the past decade, computerized decision support for diagnostic imaging has been touted as a method of improving diagnostic yield; however, only a few studies have examined outcomes of this technology in the emergency care setting. In fact, the study by Troyer et al. exemplifies the use of web-based decision support as a pretest probability intervention and specifically analyzes potential long-term benefits of such an intervention. Studies have shown the efficacy of computerized decision support in reducing unnecessary imaging.8-11 However, effectiveness of this approach outside of a clinical trial and generalizability across to other indications and settings is not as clear. Barriers to adoption of such a system have also been noted,8, 12-14 and still need to be addressed. Specifically, disruption to physician workflow,8, 13, 14 potential reduction in physician autonomy, uncertainty over optimal timing of decision support, and physician belief that their decision-making or clinical gestalt alone is appropriate or superior have been cited as barriers against adoption and acceptance. Medical decision-making is complex and requires insight into more than just the physician's pretest probability based on patient risk factors. Another component to be addressed is the physician–patient dyad and, specifically, patient preferences. This is an important component in the decision to test or not test and can influence physician behavior.15, 16 Shared decision-making, emphasized in the 2001 Institute of Medicine report, Crossing the Quality Chasm,17 is a collaborative process that takes into account the preferences and values of a patient and holds promise as a process that could be integrated into computerized decision support models. Optimizing diagnostic imaging is an important topic in emergency medicine and one in which there has been, and will continue to be, scrutiny from policymakers, providers, and patients. We welcome all who are interested in this topic to attend the 2015 Academic Emergency Medicine consensus conference, “Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization” (May 12, 2015, in San Diego, CA). The goal of this multidisciplinary conference is to generate a high-priority research agenda on which to base future research in this area. Conference proceedings will be published in the December issue of this journal (http://www.saem.org/annual-meeting/education/2015-aem-consensus-conference).