Effect Size Measures in Clinical Research

KEY POINT: Effect size measures provide important information about the quantitative relationship between variables and should generally be reported in clinical research.Related Article, see p 870 In this issue of Anesthesia & Analgesia, Yoon et al1 compare the McGrath videolaryngoscope with the Optiscope video stylet for tracheal intubation in patients with manual inline cervical stabilization. The authors report a higher first-attempt success rate with the McGrath videolaryngoscope, with an absolute difference in the success proportion of 0.11 (95% confidence interval [CI], 0.05–0.18). Intubation time was also shorter in the McGrath group, with a mean difference of 13.5 (95% CI, 5.9–21.1) minutes (Figure).Figure.: Excerpt from Yoon et al1 abstract showing the risk difference and the mean difference (with respective confidence intervals) as 2 examples of effect size measures. Group M: McGrath videolaryngoscope; Group O: Optiscope video stylet.Differences in proportions or means are 2 examples of effect size measures.2 Effect sizes describe the quantitative relationship between variables (eg, between study group allocation and the outcome). While researchers often focus on “statistical significance,” P values do not provide any information on the actual magnitude of the treatment effect.2 In fact, small and clinically meaningless differences between groups can be statistically significant, whereas important effects can fail to reach the significance threshold. Therefore, effect size estimates should typically be reported in clinical studies, as appropriately done by Yoon et al.1 These estimates should be accompanied by a CI, indicating the range of what the effect could plausibly be in the population of interest.2 Of note, while the term “effect” suggests a causal relationship, reporting of an “effect size” does not confirm a direct effect of one variable on another. The many different effect size measures can basically be divided into 2 types: (a) effect sizes that describe the strength of an association between variables,3 and (b) effect sizes that describe differences or ratios between groups.2 Correlation coefficients are examples of effect sizes that describes the strength of an association as discussed in detail in a recent Statistical Minute.4 When comparing groups on normally distributed continuous outcomes, differences in means give an intuitive understanding of the treatment effect, in particular when the measurement scale has an intrinsic meaning (eg, difference in intubation time).2 For non-normally distributed numeric outcomes, the median of differences (Hodges-Lehmann estimator) can be reported instead. Differences between groups on binary outcomes can be expressed as the absolute difference in proportions of subjects with a specific outcome (eg, successful intubation).2 As the proportion can be viewed as the estimated “risk” of having a certain outcome, this effect size is commonly referred to as risk difference. Alternatively, the risk ratio, or a closely related measure—the odds ratio—is commonly reported.5 Note that a 2% risk in one group vs 1% in the other group represents the same relative risk as a 80% vs 40% risk, while the absolute risk difference is very different (1% and 40%). To appropriately interpret relative effect measures, it is thus essential that estimates of absolute risk are also reported.