The Fragility Index: aP-value in sheep’s clothing?

This editorial refers to ‘How robust are clinical trials in heart failure?’, by K.F. Docherty et al. , doi:10.1093/eurheartj/ehw427. Randomized controlled trials (RCTs) are designed to assess objectively the safety and efficacy of a specific intervention. While public reporting of the study design, conflict of interest declarations, and the peer review process strive to fortify study findings, the veracity of published research studies has been called into question.1 Recently, the Fragility Index (FI) has been introduced as an intuitive measure of the robustness of RCTs.2 This new statistic has been recently reported in the critical care setting,3 and studied and advocated for heart failure in the paper by Docherty et al .4 In this Editorial, we will provide a brief explanation of the FI and express why caution needs to be exercised when interpreting FI. The FI is a statistical summary of an RCT that utilizes 1:1 randomization and an outcome measure that can be categorized into two levels (e.g. 30-day mortality, yes or no). When a study reaches a statistically significant difference (and the sample size per group is fixed), one can show that the number of positive responses differs between the groups. Without loss of generality, assume that a study is comprised of a novel intervention and a control condition and that fewer events will be observed in the intervention group (i.e. the intervention provides a protective effect to the patients). Suppose that the study results suggest that the intervention is protective and that the result is statistically significant. Upon careful observation, it is noted that the number of events is reasonably small and that in absolute number there is not a large difference between the two study groups. One might naturally ask the question ‘how would the results be interpreted if one of …