Nonparametric signal detectability evaluation using an exponential transformation of the FROC curve

PURPOSE: To develop an efficient nonparametric method for evaluation the detectability of signals at unknown locations in images, as a mean for image quality assessment. METHODS: We use the free-response methodology that allows the image observer to mark and score all locations found as suspicious in an image, summarizing these results in a free-response operating characteristic (FROC) curve. However, unlike the relative (or receiver) operating characteristic (ROC), or the localization ROC (LROC), the FROC curve has an undefined, theoretically infinite, right side limit. Therefore area under the FROC curves cannot be directly used as an overall performance index, as the area under the curve is for ROC or LROC. We circumvent this drawback by using a transformation of the abscissa that leads to a finite integration range. By applying an exponential transformation we derive a nonparametric estimator for such a metric, and we study its properties by deriving analytical expressions for the mean and standard deviation in conditions of scores independence. RESULTS: A comparative study with other related nonparametric estimators for ROC, LROC, and alternative FROC (AFROC) method is presented. CONCLUSIONS: The new nonparametric estimator has sensitivity and scalability properties that make it particularly advantageous for signal detectability evaluation in phantom experiments using model observers.