Statistical Comparison of Two ROC-curve Estimates Obtained from Partially-paired Datasets

The authors propose a new generalized method for ROC-curve fitting and statistical testing that allows researchers to utilize all of the data collected in an experimental comparison of two diagnostic modalities, even if some patients have not been studied with both modalities. Their new algorithm, ROCKIT, subsumes previous algorithms as special cases. It conducts all analyses available from previous ROC software and provides 95% confidence intervals for all estimates. ROCKIT was tested on more than half a million computer-simulated datasets of various sizes and configurations representing a range of population ROC curves. The algorithm successfully converged for more than 99.8% of all datasets studied. The type I error rates of the new algorithm's statistical test for differences in Az estimates were excellent for datasets typically encountered in practice, but diverged from alpha for datasets arising from some extreme situations.