What question did this study set out to answer?

The study aims to update the I2 point estimate threshold limits to enhance the detection of selection bias in RCTs.

April 25, 2026Cureus1 citationsOpen Access

The I2 Statistic As Selection Bias Test: Updated Threshold Limits

SMSteffen MickenautschUniversity of Cape Town VYVeerasamy YengopalUniversity of Cape Town

Key Points

The study aims to update the I2 point estimate threshold limits to enhance the detection of selection bias in RCTs.
Revised I2 threshold limits based on simulations and re-testing of 332 real-world RCTs.
Tested correlation between effect estimates and selection bias using Spearman's rank correlation.
Compared mean absolute risk differences between RCTs with low and high selection bias using independent samples t-test.
Increased testable RCTs from 71% to 100% with no I2 estimates exceeding revised thresholds.
Statistically significant, positive correlation (Spearman’s rho = 0.268, p < 0.0001) between absolute risk differences and selection bias levels.
Mean absolute risk difference for high bias RCTs was 0.18 (SD = 0.16) versus 0.10 (SD = 0.13) for low bias RCTs, t = -4.65; p < 0.0001.

Abstract

Aim: This study aims to revise the pre-specified I2 point estimate threshold limits of the trial-adjusted, simulated comparator trial (SCT)-based I2 test for selection bias in single randomised controlled trials (RCTs), in order to increase the percentage of testable RCTs. A further objective was to test the two null hypotheses: that, based on the test’s revised I2 point estimate thresholds, the magnitude of trial effect estimates is not significantly positively correlated with the selection bias levels (H01), and that it does not differ significantly between RCTs with identified ‘low’ and ‘high’ selection bias (H02). Methods: Revision of the I2 point estimate threshold limits was based on multiple RCT simulation and the re-testing of 332 real-world RCTs for selection bias. The selection bias level (B%) of each RCT, using the revised limits, was determined. H01 was tested using Spearman’s rank correlation, and H02 using the 2-tailed, independent samples t-test. The mean difference (MD) with 95% confidence interval (CI) between the mean absolute risk difference (RD) values with SD of RCTs with identified ‘low’ and ‘high’ selection bias was computed. Results: All 332 RCTs were testable, and none of the computed I2 point estimates fell outside the revised thresholds limits, increasing the proportion of testable RCTs from 71% to 100%. There was a statistically significant, positive, small correlation (0.1 ≤ |r| < 0.3) between the absolute RD values and established B% levels (Spearman’s rho = 0.268, p < 0.0001). The mean absolute RD for RCTs with identified ‘high’ selection bias was 0.18 (SD = 0.16), compared with 0.10 (SD = 0.13) for RCTs with ‘low’ selection bias. The difference was statistically significant: (t = -4.65; p < 0.0001; MD 0.08; 95%CI: 0.05 - 0.11). The effect size (Cohen's d = 0.52) indicates a medium effect. Both null hypotheses were rejected. Conclusion: The revised threshold limits increased the utility of the trial-adjusted, SCT based I2 test from 71 to 100% testable RCTs. The rejection of both null hypotheses indicates that the previously established statistically significant, positive relationship between selection bias levels (B%) and effect estimates (absolute RD) is independent of the changes in the I² threshold limits.

Demander à l'IA

Bookmark

View Full Paper