The Use of Continuous and Discrete Confidence Judgments in Receiver Operating Characteristic Studies of Diagnostic Imaging Techniques

Virtually all receiver operating characteristic (ROC) studies of medical imaging methods have involved data collected on a discrete confidence-rating scale, though in principle ROC studies can be conducted with either discrete or continuous scales. Continuous scales provide potential advantages in medical applications of ROC analysis, but the possible dependence of ROC results on the kind of scale used in an observer-performance experiment has not been investigated empirically. The authors conducted a multi-observer ROC study in which radiologists read abdominal computed tomography (CT) examination results displayed on a workstation with a high-resolution video monitor. Ninety-five examinations were evaluated by five radiologists who reported their confidence concerning the presence of abdominal masses on a conventional five-category discrete scale and, in a separate reading session, on a continuous subjective-probability scale. ROC analysis showed no statistically significant difference between the detection accuracies (Az) achieved with the two scales by any reader. Accuracies averaged over all readers were identical with the two scales. The results of this study suggest that discrete or continuous scales often can be used interchangeably in image-evaluation studies. However, the authors recommend continuous scales for routine use in radiologic ROC studies because of their potential advantages in some situations.