Comparison of bias adjustment in meta-analysis using data-based and opinion-based methods

Introduction: Several methods exist for bias adjustment of meta-analysis results, but there has been no comprehensive comparison with unadjusted methods. We compare 6 bias-adjustment methods with 2 unadjusted methods to examine how these different methods perform. Methods: We re-analyzed a meta-analysis that included 10 randomized controlled trials. Two data-based methods (Welton’s data-based approach and Doi’s quality effects model) and 4 opinion-informed methods (opinion-based approach, opinion-based distributions combined statistically with data-based distributions, numerical opinions informed by data-based distributions, and opinions obtained by selecting areas from data-based distributions) were used to incorporate methodological quality information into the meta-analytical estimates. The results of these 6 methods were compared with 2 unadjusted models: the DerSimonian-Laird random effects model and Doi’s inverse variance heterogeneity model. Results: The 4 opinion-based methods returned the random effects model estimates with wider uncertainty. The data-based and quality effects methods returned different results and aligned with the inverse variance heterogeneity method with some minor downward bias adjustment. Conclusion: Opinion-based methods seem to only add uncertainty rather than bias adjust.