Key result
In comparative studies of single-row versus double-row rotator cuff repair, the median fragility index was 2, and 81% of statistically significant outcomes required only a single event reversal to lose significance.
Why the study?
P values are vulnerable to a small number of outcome reversals, so this study evaluated the statistical stability of studies comparing single-row to double-row rotator cuff repair.
Systematic Review (n=22)
Effect estimate: Median FI 2; Median FQ 0.035
The statistical significance of comparative studies evaluating single-row versus double-row rotator cuff repair is highly fragile, often relying on fewer event reversals than the number of patients lost to follow-up.
Clinicians should interpret single- versus double-row rotator cuff repair evidence skeptically; challenges robustness of many comparative study conclusions.
Background: Comparative studies and randomized controlled trials (RCTs) often use the P (probability) value to convey the statistical significance of their findings. P values are an imperfect measure, however, and are vulnerable to a small number of outcome reversals to alter statistical significance. The inclusion of a fragility index (FI) and fragility quotient (FQ) may aid in the interpretation of a study’s statistical strength. Purpose/Hypothesis: The purpose of this study was to examine the statistical stability of studies comparing single-row to double-row rotator cuff repair. It was hypothesized that the findings of these studies would be vulnerable to a small number of outcome event reversals, often fewer than the number of patients lost to follow-up. Study Design: Systematic review; Level of evidence, 3. Methods: We analyzed comparative studies and RCTs on primary single-row versus double-row rotator cuff repair that were published between 2000 and 2021 in 10 leading orthopaedic journals. Statistical significance was defined as a P < .05. The FI for each outcome was determined by the number of event reversals necessary to alter significance. The FQ was calculated by dividing the FI by the respective sample size. Results: Of 4896 studies screened, 22 comparative studies, 10 of which were RCTs, were ultimately included for analysis. A total of 74 outcomes were examined. Overall, the median FI was 2 (interquartile range [IQR], 1-3), and the median FQ was 0.035 (IQR, 0.020-0.057). The mean FI was 2.55 ± 1.29, and the mean FQ was 0.043 ± 0.027. In 64% of outcomes, the FI was less than the number of patients lost to follow-up.) Additionally, 81% of significant outcomes needed just a single outcome reversal to lose their significance. Conclusion: Over half of the studies currently used to guide clinical practice have a number of patients lost to follow-up greater than their FI. The results of these studies should be interpreted within the context of these limitations. Future analyses may benefit from the inclusion of the FI and the FQ in their statistical analyses.
No takes yet. Share an insight, caveat, or question.
Fackler et al. (2022) conducted a systematic review in Rotator cuff tears (n=22). Single-row versus double-row anchoring for rotator cuff repair was evaluated on Fragility Index (FI) and Fragility Quotient (FQ) (Median FI 2; Median FQ 0.035). In comparative studies of single-row versus double-row rotator cuff repair, the median fragility index was 2, and 81% of statistically significant outcomes required only a single event reversal to lose significance.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: