In Response: I agree with Drs. Overdyk and Roy that "in magic tricks, what you don't see is as important as what you do see." Accordingly, the innuendo that studies not included in my analysis [1] would, if included, weaken or falsify its conclusion should be examined. Taylor et al. [2] did not report incidence of emesis, but they did report nausea scores. The group receiving N2 O experienced more nausea than the non-N2 O group (8 vs 4). My study was limited to incidence of emesis, but if nausea had been an inclusion criterion, this study would have supported the inference that N2 O has an adverse effect. The Methods section of the study by Jensen et al. [3] states that they counted the frequency of requests for "treatment for nausea." In their Results section, that variable is labeled "requirements for anti-emetic therapy" [3]. These authors found an absolutely higher incidence of administration of such therapy in their N2 O group, but, as with the study by Taylor et al. [2], I could not include this study in my analysis because it did not report the incidence of emesis. Again, if I had included this study, the inference from my analysis would have been strengthened. In distinction to the above papers, I missed the study by Pedersen et al. [4]. They did report the incidence of emesis, and they found a higher incidence (71%) in their N2 O group than in their non-N2 O group (42%). Had I found this study, it would have been included in my analysis and it would have strengthened the inference that N2 O increases the incidence of vomiting. I also missed the abstract by Arellano et al. [5]. These investigators found a higher incidence of emesis in their non-N2 O group (two patients out of 68) than in their N2 O group (one patient out of 70). While this paper is excluded by Overdyk and Roy because it "involved total intravenous anesthesia with propofol … a widely accepted anti-emetic," according to the criteria of my analysis, it would have been included. Had it been, its effect would have largely offset the effect of Pedersen et al., and the statistical significance of my result would remain P < 0.00005. Somewhat more interesting is Overdyk and Roy's conjecture that "100% oxygen is an anti-emetic." However, their proffered empirical support involves three magic tricks. First is the use of a one-tailed Fisher's exact test. This is seldom justified, and cannot be justified when the hypothesis being tested is a novel conjecture about what "appears to be an inconsequential variable at first." Here's why. Suppose I offered to sell you a "loaded" penny for one dollar. You want evidence that it really is loaded, so I flip it four times, get four heads, and say, "P = 0.0625-close enough; I'm only asking for a buck." The trick is that I did not initially say, "This coin is loaded toward heads"-I just said, "This coin is loaded"-and an ordinary penny is as likely to come up four tails as four heads, doubling the probability that you would be buying a plain penny for a dollar. This is why one-tailed tests are frowned upon [6]. (I used a two-tailed test in my analysis, even though the conjecture that N2 O is antiemetic [the other side of the coin] does not enjoy widespread popularity.) Overdyk and Roy's second trick is their inclusion of Jensen'93 and Taylor'93, neither of which report the incidence of emesis. And their third trick is exclusion of Arellano '94 and Lim '93-especially tricky in light of Overdyk and Roy's insistence upon "anesthetic regimens differing only by N2 O omission" (a criterion that both of these studies satisfy). Doing the proper Fisher's exact test brings the P value of Overdyk and Roy's analysis up to 0.1. Excluding Jensen '93 and Taylor '93 brings it to 0.2. Including Arellano '94 and Lim '93 brings it to 0.21. Would you buy a purportedly loaded penny because it landed on the same side three times in a row? If so, please send one dollar and a stamped, self-addressed envelope to the address below-I've got your coin! Meanwhile, it seems fair to assume that readers who do not send money agree that people who live in glass houses should not throw stones. John D. Hartung, PhD Department of Anesthesiology; State University of New York-Health Science Center at Brooklyn; Brooklyn, NY 11203-2098
No takes yet. Share an insight, caveat, or question.
John D. Hartung (1997) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: