The fragility of randomized controlled trials in critical care

Studies conducted in the field of critical care medicine often investigate short-term binary outcomes that require large sample sizes to reach statistical power. The Fragility Index is a statistical measure of the number of events needed to convert a statistically significant difference between groups into a difference that is no longer statistically significant. This study aimed to systematically review the robustness of published randomized clinical trials (RCTs) in critical care and the methodological pitfalls that contribute to a lower Fragility Index. We used Ovid/MEDLINE and PubMed to identify all RCTs in the top ten critical care medicine journals from January 1, 2020 to May 31, 2025. Eligible studies reported a statistically significant primary binary outcome. Our search yielded 815 studies, of which 59 studies were eligible and included in the analysis. We calculated the Fragility Index, extracted other trial methodological characteristics (expected and observed event rates, attrition rates, effect sizes, and sample size), including methods used by the authors to reduce risks to internal validity. We then analyzed the relationships between these characteristics and the Fragility Index. The median Fragility Index was 6 (range, 0–73; interquartile range, 2–10). The median expected relative risk reduction was 50% (IQR, 36%-65%). 23/53 (43%) studies that reported expected event rates, achieved an event rate in the experimental arm at or above the expected rate used for planning. 28/59 (47%) studies had a Fragility Index less than or equal to the number of participants lost to follow-up. The median Fragility Index in the critical care literature has increased, but differences in outcome selection may explain this finding. The Fragility Index is a simple and conservative starting point to evaluate the robustness of the findings of a trial. In addition to threats to validity, other threats to robustness include overestimating the expected effect size during the planning phase of the trial, recruiting a sample with too few events, and suffering a high rate of attrition.