RADIFUSION: a multi-radiomics deep learning based breast cancer risk prediction model using sequential mammographic images with image attention and bilateral asymmetry refinement

Objective: Breast cancer is a significant public health concern, and early detection is critical for triaging high-risk patients. Sequential screening mammograms can provide important spatiotemporal information about changes in breast tissue over time, which may be useful for breast cancer risk prediction models. Approach: In this study, we propose a deep learning architecture called Radiomics Fused Gated Attention (RADIFUSION) that utilizes sequential mammograms and incorporates a linear image attention mechanism, radiomics features, a new gating mechanism to combine different mammographic views, and bilateral asymmetry-based finetuning for breast cancer risk assessment. We evaluate our model on a screening dataset called the Cohort of Screen-Aged Women (CSAW), consisting of 8,723 patients altogether. Main results: Based on results obtained on the independent testing set consisting of 1,749 women, our approach achieved slightly better performance compared to other state-of-the-art models with area under the receiver operating characteristic curves (AUCs) of 0.905, 0.872 and 0.866 in the three respective metrics of 1-year AUC, 2-year AUC and 3-year AUC. Our study highlights the importance of incorporating various deep learning mechanisms, such as image attention, radiomics features, a gating mechanism, and bilateral asymmetry-based fine-tuning, to improve the accuracy of breast cancer risk assessment. We also demonstrate that our model's performance was enhanced by leveraging spatiotemporal information from sequential mammograms. Significance: Our findings suggest that RADIFUSION can provide clinicians with a useful tool for breast cancer risk assessment.