F*, an interpretable transformation of the F measure, equates to the critical success index

When analysing large datasets, suitable outcome measures are needed. For example, in clinical medicine, we often want to know how well a test identifies people with or without a particular diagnosis. Traditionally, this evaluation has been calculated in terms of Sensitivity and Specificity, and Positive and Negative Predictive Values (PPV, NPV), but many other test outcome measures are available. Recently, a measure termed F* was described, as an interpretable transformation of the F measure. The F measure is a unitary metric calculated as the harmonic mean of Sensitivity and PPV. We show that F* is in fact identical to a previously described measure which is monotonically related to the F measure, and which has been variously named in previous publications as the ratio of verification, the Jaccard similarity measure or index, the threat score, the Tanimoto index, and the critical success index (CSI). The origins of these different terms, which date from the late 19th–20th century, in different scientific disciplines (weather forecasting, ecology, machine learning) may explain the repeated independent redescription of this measure. More recently, the advantages of applying CSI, a metric routinely used to verify weather forecast accuracy, have been demonstrated in medical diagnostic accuracy studies.