Effect of independent multiple reading of mammograms on detection performance

The goal of this study was to assess the effect of independent combination of multiple readers in mammography on detection performance, using different rules to combine localized observer responses. A group of 12 radiologists each read a series of 192 screening mammograms, including 96 prior mammograms of breast cancer cases in which a visible sign of abnormality could be identifed in retrospect. The other 96 cases were normal. In total the 12 readers annotated 1890 findings. LROC analysis was used to measure performance. The mean sensitivity in a false positive interval from 2 to 8% was 31.4% for single reading (range: 14.4% - 46.9%). The best rule for combination of observer scores was taking the average of all radiologists, using a zero score for radiologists who did not annotate the finding. With this strategy the average performance of 2 readers combined, in the interval selected, went up to 42.2%. When the interpretations of more readers were independently combined the mean sensitivity further increased, up to a level of 64.8% for the combination of all 12 readers. Using the mean score of only those readers who reported a finding turned out to be a poor strategy, yielding results that were similar or worse than single reading.