Comparing percent breast density assessments of an AI-based method with expert reader estimates: inter-observer variability

Breast density assessment is an important part of breast cancer risk assessment, as it has been known to correlate with risk. Mammograms would typically be assessed for density by multiple expert readers, however, interobserver variability can be high. Meanwhile, automatic breast density assessment tools are becoming more prevalent, particularly those based on artificial intelligence. We evaluate one such method against expert readers. A cohort of 1329 women going through screening was used to compare between two expert readers selected from a pool of 19, and a single such reader versus a deep learning based model. Whilst the mean differences for the two experiments were statistically similar, the limits of agreement between the AI method and a single reader are substantially lower at +SD 21 (95% CI : 20.07, 22.13) -SD 22 (95% CI : -22.95, -20.90) against +SD 31 (95% CI : 33.09, 28.91) -SD 28 (95% CI : -30.09, -25.91) between two expert readers. Additionally, the absolute intraclass correlation coefficients (two-way random multiple measures) were 0.86 (95% CI : 0.85, 0.88) between the AI and reader and 0.77 (95% CI : 0.75, 0.80) between the two readers achieving statistical significance. Our AI-driven breast density assessment tool has better inter-observer agreement with a randomly selected expert reader than two expert readers (drawn from a pool) do with one another. Additionally, the automatic method has similar inter-view agreement to experts and maintains consistency across density quartiles. Deep learning enabled density methods can offer a solution to the reader bias issue and provide consistent density scores.