A55-40 Effect of Adjusting for Trial Site on Observed Treatment Differences in Pragmatic Trials of Emergency Tracheal Intubation

Rationale Trialists must decide whether to adjust for trial site when analyzing multicenter trials. Including trial site as a random effect in the analysis of a multicenter trial accounts for differences in patient populations and clinical practice patterns between sites (correlation) with the aim of controlling for heterogeneity across sites, improving precision, and supporting valid statistical inference across the full population. These potential benefits, however, may come at the cost of increased statistical complexity and difficulty interpreting results. The effect of accounting for trial site on estimates of treatment effect in critical care trials has not been empirically examined previously. Methods We performed a secondary analysis of the data from six pragmatic multicenter randomized trials of emergency tracheal intubation. We compared the effect of trial group assignment on the primary outcome of each trial using (i) unadjusted analysis (Chi-square test for categorical variables or Wilcoxon rank-sum for continuous variables), (ii) a model without a random effect for site, and (iii) a model with a random effect for site (generalized linear mixed-effects model with a logit link with a random effect for site and a fixed effect for group assignment for categorical variables or cumulative link mixed model with logit link with a random effect for site and a fixed effect for group assignment for continuous variables). Results Six randomized trials were included in the analysis: PREOXI (1,301 patients at 24 sites); DEVICE (1,417 patients at 17 sites); BOUGIE (1,102 patients at 15 sites); PREPARE II (1,065 patients at 11 sites); PreVent (401 patients at 7 sites); CHECK UP (262 patients at 4 sites). Among the 6 trials, the median difference in estimated magnitude of the absolute treatment effect between the unadjusted analysis and the analysis with trial site as a random effect was 0.15 percentage points. Comparisons of the 95% confidence intervals and p-values are shown in Table 1. For none of the trials did inclusion of a random effect for trial site change the statistical significance or the interpretation of the results. Conclusion In this secondary analysis of six pragmatic trials of emergency tracheal intubation, inclusion of trial site as a random effect in the analysis of the primary outcome did not substantively change the estimated treatment effects, 95% confidence intervals, p-values, or interpretation of statistical or clinical significance. These findings suggest that the benefits of accounting for trial site in multicenter trials may be small in some contexts. This abstract is funded by: None