Impact of Artificial Intelligence on Mammography Reader Performance Across Training Levels: A Pilot Study in Screening-Naive Population

This article determines the impact of artificial intelligence (AI)-assisted mammographic interpretation on reader performance across different expertise levels in a diagnostic setting within a screening-naive population. In this prospective, single-center diagnostic accuracy study (January 2024), 240 mammographic studies (358 breasts) were included. Four readers (general radiologist, senior resident, junior resident, technologist) interpreted 180 breast images with AI assistance and 178 without AI. Two commercial AI tools (MammoScreen and INSIGHT MMG) were used. BI-RADS (Breast Imaging-Reporting and Data System) scores (1–5) were independently assigned by readers and dichotomized as malignant (4–5) versus nonmalignant (1–3). Reference standard was the double-read report by a breast specialist. Diagnostic metrics (sensitivity, specificity, accuracy; Wilson 95% confidence interval [CI]) were compared using two-sample Newcombe tests with Bonferroni correction. Fixed-effects logistic model estimated the overall AI effect. Agreement was measured with Cohen's κ and Fleiss' κ. Malignancy prevalence was 18% (with-AI) versus 15% (without-AI) (p = 0.34). The technologist's accuracy improved from 0.83 (0.77–0.89) to 0.92 (0.86–0.95) with AI (p_adj = 0.041). No significant accuracy changes were seen in the other readers (p_adj ≥ 0.296). Overall AI effect was nonsignificant (odds ratio = 1.31; 95% CI, 0.81–2.12; p = 0.28). Cohen's κ improved for the junior resident (0.59–0.71) and technologist (0.45–0.74); Fleiss' κ increased from 0.49 to 0.65. AI assistance improved sensitivity, accuracy, and agreement for less-experienced readers while maintaining stable performance for experienced readers, supporting its use in resource-limited, screening-naive settings.