A residual-attention multimodal fusion network (ResAMF-Net) for detection and classification of breast cancer

Digital breast tomosynthesis (DBT), synthetic mammography, and full-field digital mammography (FFDM) are commonly used medical imaging modalities for breast cancer screening. Due to the data complexity, most CAD research applies to only one modality, which under-utilizes the complementary information in these 2D and 3D modalities. In this study, we propose a Residual-Attention Multimodal Fusion network (ResAMF-Net) that integrates lesion features across these modalities. We evaluated network performance on a large private dataset, which contains 769 cancer cases and 1375 noncancer cases (including 362 benign and 1013 normal cases) for a total of 2144 cases. At 90% case sensitivity, ResAMF-Net increases specificity by 8%, which can substantially improve radiologist workflow because almost all screening cases are true negatives.