Assessing the Ability of Randomized Controlled Trials to Inform Clinical Practice in Surgery

Background: Randomized controlled trials (RCTs) often fail to convey relevant and actionable results. We hypothesized that these shortcomings reflect deficiencies in trial conception and planning. We aimed to develop an appraisal tool to examine the extent to which RCTs incorporate core elements required to meaningfully inform clinical practice and assess a set of contemporary surgical trials. Methods: This cross-sectional study involved the development and use of an original appraisal tool assessing the ability of RCTs to inform clinical practice. The tool was pilot-tested extensively and subsequently applied to an unselected sample of consecutively published surgical RCTs from 2024. Data were extracted from published articles and corresponding pretrial documentation. We defined that an RCT was potentially clinically informative by fulfillment of 4 criteria: (1) report of specific primary endpoint(s); (2) alignment of reported trial hypothesis with trial design; (3) use of a clinically relevant effect size informing sample size calculation; and (4) consideration of patient-relevant clinical importance in interpreting results. Results: In the main study, only 1 out of 167 RCTs (0.6%, Wilson 95% confidence interval: 0.1% to 3.3%) fulfilled all criteria. Common limitations were lack of considering clinical relevance in sample size calculations and results interpretation. Discrepancies between pretrial documents and final publications mainly concerned primary endpoints. Conclusions: We developed a novel tool to assess the potential of RCTs to inform clinical practice, complementing established tools for risk of bias and reporting quality. Our findings raise concerns about the clinical relevance of surgical RCTs.