Walk a Mile in My Shoes

A significant fraction of the articles in Anesthesia & Analgesia report the results of clinical trials funded by government agencies (e.g., National Institutes of Health), the pharmaceutical industry, or by the institution in which the investigators work. The design of clinical trials is a challenging process in which investigators balance their desire to answer many questions with high levels of certainty against the realities of funding, ethical considerations, magnitude of effort, and other practicalities. As pointed out previously, the perfect trial cannot be performed.1 All trials are imperfect, and some are even highly flawed, in that the trial design cannot answer any question in a meaningful way. This issue of Anesthesia & Analgesia contains an interesting and likely controversial article addressing trial design. Deans et al.2 observe that some published clinical trials have “practice misalignments,” limiting the utility of the results. They define practice misalignment as the difference between the treatment specified by one or more arms of the study and prevailing medical practice. They pose a fair question: “If neither arm reflects clinical practice, then how can one interpret the study results relative to current practice?” Practice misalignments often exist in clinical trials because of the constituencies for whom the trials are conducted. Many clinical trials are conducted by the pharmaceutical industry to meet Food and Drug Administration (FDA) requirements to market drugs in the United States. By definition, these must be “adequate and well controlled trials” that convince the FDA that the drug works. If the new therapy is compared with prevailing practice and there is no difference between treatments, one possible conclusion is that the drug works as well as the prevailing practice. Unfortunately, there are 2 other possible explanations: neither drug works as given in the trial, or the study was underpowered to show differences. Because the FDA wants to be certain that the drug is effective, the agency typically prefers trials demonstrating superiority over placebo. This causes practice misalignments but meets the FDA's requirement that the pharmaceutical manufacturer demonstrates the claimed efficacy. Unless the FDA changes its requirement for demonstration of efficacy, trials sponsored by the pharmaceutical industry are likely to use placebo groups unless inclusion of such groups would be considered unethical (e.g., giving surgical patients a placebo in lieu of a proper anesthetic). Placebo studies for drug registration are an extreme example of studies with practice misalignments, and Deans et al.2 agree, “for conditions with no proven therapy or where withholding therapy is acceptable, use of a placebo control may represent the best design option.” The lack of a control arm aligned to current practice often makes the placebo controlled clinical trials required by the FDA completely uninteresting to journals, because it is expected that a “me too” drug works better than placebo. In this case, a study with proper practice alignment, i.e., a control group reflecting current practice, will be much more publishable. Practice misalignments are not easily eliminated in part because it is often difficult to define prevailing practice. Consider the study on low-volume versus high-volume mechanical ventilation (the “ARMA” trial3) cited by Deans et al. In designing the study, the investigators (in consultation with the National Institutes of Health) spent more than a year considering trial design, including incorporating an arm in which tidal volume was adjusted based on lung compliance (personal communication, Michael Matthay, MD, University of California, San Francisco, April, 2009). Despite this considerable diligence by expert investigators, Deans et al. conclude that “the results of the ARMA trial were confounded by the presence of practice misalignments.” As Deans et al. note, to eliminate practice misalignment while retaining study power, it is likely necessary to add more groups. We can perform a “back of the envelope” power analysis to determine the minimal sample size that will yield statistical success. Increasing the number of groups from 2 to 3 and assuming the same effect size for this new group obviously increases the number of subjects by 50% (i.e., 3 groups of the same size rather than 2). However, when results from the control group are compared with >1 group (in this case, the 2 treatment groups), there is a statistical penalty for multiple comparisons.* One such penalty, the Bonferroni correction, requires that each statistical claim be evaluated at the 2.5% level (5%/2). To achieve this level of statistical success, a power analysis would reveal the need for an even larger increase in sample size. Under these circumstances, adding a third group would require a 72% increase in sample size, lengthening conduct of a trial, and increasing cost. When Matthay and colleagues evaluated the impact of adding a third group, they determined that duration of the clinical trial would increase by 7 years, far too long to be practical. A related issue is the expectations of journal editors and reviewers regarding publication of clinical trials. Satisfying regulatory requirements for drug approval may not be sufficient to warrant publication of a clinical trial. For example, a clinical trial might not offer any new insights, or reviewers may consider that the trial has not explored utility of the treatment sufficiently. Consider a trial evaluating the bioequivalence of 2 preparations: the original branded drug, and a generic drug. The FDA requires such trials, but they contain no novel information. Journals typically have no interest in publishing such research. A more borderline case is a trial that only considers a single dose. In such trials, one is unable to assess whether that dose is optimal. A trial establishing the dose versus response relationship for the drug will help guide future therapy, even if none of the doses in the trial conforms to current clinical practice (and thus the trial is “misaligned”). If one wants to guide clinicians who must choose between 2 drugs, then one should study 2 drugs, incorporate dose versus response curves for both, establish therapeutic windows for both drugs, and compare their efficacy and toxicity across the full spectrum of exposure. Many arms in that trial would be misaligned with current practice, but the wealth of information would clearly guide future practice. Such trials are incredibly difficult, very expensive, take a long time to complete, and risk becoming irrelevant before the trial is completed. Deans et al. primarily focus on safety, not generalizability, in their discussion. It is self-evident that one should not propose a treatment in a clinical trial well beyond currently accepted therapy, exposing patients to Rumsfeldian unknown unknowns. We agree completely. However, one must decouple lockstep titration of therapy to disease severity, because otherwise the effects of therapy are necessarily, and by design, confounded by disease severity. For example, if one increases the dose of fentanyl in response to pain, then an uniformed analysis of the study result might conclude that fentanyl causes pain. While the error is obvious in this example, in many studies the relationships are complex, and lockstep titration of therapy to disease may compromise the ability of the data analyst to reach useful conclusions. Deans et al. are correct: practice misalignments exist in clinical studies, and these misalignments impair interpretation and generalizability of study results to clinical practice. However, the challenge faced by individual investigators trying to answer complex questions must not be underestimated. The most efficient design to answer the question may be a fixed-dose clinical trial, despite practice misalignment. We have done dozens of such clinical trials to address important questions in clinical pharmacology. Virtually all of these trials have had practice misalignments. Before dismissing the value of these trials, walk a mile in our shoes. ACKNOWLEDGMENTS Dr. Shafer is the Editor-in-Chief of Anesthesia & Analgesia. The views in this editorial are the personal perspectives of the authors. The final disposition of this Editorial was handled by Dr. Jukka Takala, section Editor for Critical Care. The title of the editorial is taken from the song by Joe South, “Walk a Mile in My Shoes,” Lowery Music, 1974.