What question did this study set out to answer?

To evaluate the statistical performance of univariate and dependency-adjusted meta-analytic methods designed to detect selective reporting and publication bias when effect sizes are correlated.

July 16, 2020Psychological Methods

Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes.

View Full Paper

Ask AI

Bookmark

Authors

MRMelissa A. RodgersAmerican Institutes for Research JPJames E. PustejovskyUniversity of Wisconsin–Madison

Discussion

Member takes

Implication

Monte Carlo simulation study demonstrates that ignoring effect size dependence inflates false positives in selective reporting tests, highlighting the necessity of robust multilevel modeling.

Key Points

To evaluate the statistical performance of univariate and dependency-adjusted meta-analytic methods designed to detect selective reporting and publication bias when effect sizes are correlated.
Conducted Monte Carlo simulations to assess univariate small-study effect tests, including the trim and fill method, Egger's regression test, and a three-parameter selection model (3PSM) likelihood ratio test.
Tested performance when effect size dependencies were ignored, handled with ad hoc procedures, or explicitly modeled via Egger's regression with robust variance estimation (RVE) or multilevel meta-analysis (MLMA).
Ignoring effect size dependency inflated Type I error rates across all evaluated univariate selective reporting tests.
Egger's regression variants incorporating RVE or MLMA maintained appropriate Type I error rates, whereas the 3PSM likelihood ratio test failed to consistently control Type I errors.
With the exception of the 3PSM, all evaluated methods demonstrated limited statistical power to detect selection bias unless selection for statistically significant findings was strong.

Cite This Study

Rodgers et al. (2020) studied this question.

synapsesocial.com/papers/69d702bd8dca315383ed9776 https://doi.org/10.1037/met0000300

View Full Paper

Ask AI

Bookmark

Implication

Monte Carlo simulation study demonstrates that ignoring effect size dependence inflates false positives in selective reporting tests, highlighting the necessity of robust multilevel modeling.

Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes.

Discussion

Member takes

Implication

Key Points

Cite This Study

Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes.

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider