A Nationwide Study of Interobserver Agreement and Accuracy in HER2 Immunohistochemistry and the Impact of Artificial Intelligence on HER2 Interpretation

Purpose: Accurate assessment of low levels of human epidermal growth factor receptor 2 (HER2) expression has become increasingly important with the expansion of HER2-targeted therapies to patients with HER2-low and HER2-ultralow breast cancer (BC). Following the introduction of revised College of American Pathologists (CAP) guidelines incorporating the HER2-ultralow (0+) category, we conducted a nationwide multireader study to evaluate interobserver variation in HER2 interpretation and to investigate the impact of artificial intelligence (AI)-assisted image analysis. Materials and Methods: A total of 168 BC cases stained using the VENTANA HER2 (4B5) assay were digitized as whole-slide images (WSIs). Thirty-four board-certified breast pathologists independently reviewed all cases without AI assistance and repeated the assessment after a four-week washout period using AI support. Interobserver agreement, classification performance, and reading time were compared between the two phases. Results: AI assistance significantly improved interobserver agreement (Krippendorff's alpha, 0.7415 to 0.8849; Fleiss' kappa, 0.4492 to 0.7524; p<0.001). The largest gains in diagnostic performance were observed in HER2-ultralow and HER2 IHC 1+ BCs, with sensitivities improving from 48.5% to 73.9% and from 64.9% to 87.7%, respectively (p<0.001). Although overall reading time was not reduced in the unadjusted analysis, AI assistance shortened interpretation time in HER2-ultralow BCs by 3.12 seconds (5.1% reduction). Conclusion: AI-assisted assessment improved the reproducibility and diagnostic performance of HER2 interpretation, with the greatest benefit observed in clinically relevant HER2 low-expression categories. The reduced interpretation time in HER2-ultralow BCs further suggests that AI can improve efficiency in diagnostically challenging cases encountered in routine practice.