“Crude Vaccine Effectiveness” Is a Misleading Term in Test-negative Studies of Influenza Vaccine Effectiveness

To the Editor: We have been reading with interest the recent reports of interim estimates of influenza vaccine effectiveness for the 2014/2015 northern hemisphere season.1–4 The test-negative design has now become established as the standard observational design for timely assessment of influenza vaccine effectiveness.5 Studies following this design often report both a crude and adjusted vaccine effectiveness estimate. However, it is important to consider our ultimate goal in reporting vaccine effectiveness estimates and whether an unadjusted estimate is useful. The term “vaccine effectiveness” implies a causal effect rather than a mere correlation, and estimation of a causal effect requires confounding to be addressed.6 In a typical test-negative study, which is similar to a case–control study, patients with influenza-like illness are enrolled in a clinical setting and tested for influenza. The crude odds ratio is obtained by dividing the odds of vaccination among influenza-positive patients by the odds of vaccination among influenza-negative patients. This measure indicates the correlation of vaccination with influenza, but may not be an accurate estimate of the causal effect of vaccination on the risk of influenza because that association may be confounded. Confounding variables are associated with, but not the result of, both the exposure and the outcome, conditional on all other variables.7 In observational studies, statistical adjustment of estimates (e.g., regression or stratification) is usually necessary to overcome confounding and ensure exchangeability between groups. This adjusted estimate will approximate the causal effect, such as the effectiveness of a vaccine. In observational studies of vaccine effectiveness, including the test-negative design, vaccine effectiveness is commonly calculated as 1 − ORadj × 100%.8 Absent other biases, the difference between the crude odds ratio and the adjusted odds ratio should show the degree of bias caused by confounding. It can therefore be worthwhile to present the crude odds ratio and the adjusted odds ratio. However, we propose here that it is improper to present a “crude vaccine effectiveness,” since this value has no causal interpretation and should not be presented as an estimate of vaccine effectiveness. Similarly, it should be unnecessary to use the word “adjusted” when reporting vaccine effectiveness because adjustment should have been performed to present an estimate of a causal effect. We recognize that the differences in crude and adjusted odds ratios may be caused by other biases, including sparse data bias, measurement error, or residual confounding. Nevertheless, when reporting an estimate with an implicit causal effect, it can be misleading to report a crude estimate. Sheena G. Sullivan WHO Collaborating Centre for Reference and Research on Influenza Peter Doherty Institute for Infection and Immunity Melbourne, VIC, Australia Fielding School of Public Health University of California Los Angeles, CA, USA [email protected] Benjamin J.