Small randomized controlled trials (RCTs) in COVID-19 meta-analyses have been associated with more favourable treatment effects and reduced result stability. This study assessed how trial size impacts effect estimates, statistical stability, and risk of bias. Following PRISMA guidelines, we identified meta-analyses of COVID-19 treatments included in WHO, NIH, and the LIVING Project. Trials were classified by log-scale sample size, and separate pooled meta-analyses were conducted for large-only, small-only, and combined trials. Comparative metrics included the Ratio of Odds Ratios (ROR), Kappa statistics, Fragility Index (FI), Reverse Fragility Index (RFI), and Cochrane Risk of Bias assessments. Sensitivity analyses applied alternative size thresholds (≥ 1000 participants and median-based cutoffs) and stratified results by treatment and outcome type. Across 25 meta-analyses including 221 RCTs (46 large, 175 small), small trials produced more extreme estimates in 19 analyses and wider confidence intervals in 23. The pooled ROR was 0.85 (95% CI: 0.76-0.95; P = 0.004), decreasing to 0.81 (95% CI: 0.68-0.95; P = 0.011) when limited to small trials published before the first large trial. RORs remained below 1 across treatment and outcome types. Agreement between small and large trials was minimal, while large trials showed substantial agreement with overall estimates. Stability and bias profiles favoured large trials (FI: 14.0 vs. 4.0; RFI: 10.0 vs. 5.0). In conclusion, small RCTs tend to overestimate treatment effects and yield less precise, less stable results. Meta-analyses should prioritise large, high-quality trials and interpret small-study findings with caution, particularly in rapidly evolving research contexts.
Kim et al. (Sun,) studied this question.