Bias resulting from the use of ‘assay sensitivity’ as an inclusion criterion for meta‐analysis

Assay sensitivity has been proposed as a criterion for including psychiatric clinical outcome studies in meta-analyses. The authors assess the performance of assay sensitivity as a method for determining study appropriateness for meta-analysis by calculating expected standard drug vs placebo effect sizes for various combinations of high quality and flawed studies. In the absence of flawed studies, expected effect sizes are close to unbiased only when sample sizes are very large. In the presence of flawed studies, expected effect sizes tend to be substantially biased except under simultaneous conditions of high power, a large proportion of flawed studies, and a population standard vs placebo effect size of flawed studies considerably lower than that of high quality studies. The authors conclude that this method is not robust and can lead to serious bias. Unless it can be shown that specific conditions hold, assay sensitivity should not be used to make quality judgments of studies.