Resampling Techniques in the Analysis of Non-binormal ROC Data

The methods most commonly used for analyzing receiver operating characteristic (ROC) data incorporate "binormal" assumptions about the latent frequency distributions of test results. Although these assumptions have proved robust to a wide variety of actual frequency distributions, some data sets do not "fit" the binormal model. In such cases, resampling techniques such as the jackknife and the bootstrap provide versatile, distribution-indepen dent, and more appropriate methods for hypothesis testing. This article describes the ap plication of resampling techniques to ROC data for which the binormal assumptions are not appropriate, and suggests that the bootstrap may be especially helpful in determining con fidence intervals from small data samples. The widespread availability of ever-faster com puters has made resampling methods increasingly accessible and convenient tools for data analysis. Key words: receiver operating characteristic; ROC; resampling; jackknife; bootstrap; diagnostic testing; diagnostic information; confidence interval. (Med Decis Making 1995;15:358-366)