YOLO-based CAD framework with ViT transformer for breast mass detection and classification in CESM and FFDM images

Abstract Breast cancer detection is considered a challenging task for the average experienced radiologist due to the variation of the lesions’ size and shape, especially with the existence of high fibro-glandular tissues. The revolution of deep learning and computer vision contributes recently in introducing systems that can provide an automated diagnosis for breast cancer that can act as a second opinion for doctors/radiologists. The most of previously proposed deep learning-based Computer-Aided Diagnosis (CAD) systems mainly utilized Convolutional Neural Networks (CNN) that focuses on local features. Recently, vision transformers (ViT) have shown great potential in image classification tasks due to its ability in learning the local and global spatial features. This paper proposes a fully automated CAD framework based on YOLOv4 network and ViT transformers for mass detection and classification of Contrast Enhanced Spectral Mammography (CESM) images. CESM is an evolution type of Full Field Digital Mammography (FFDM) images that provides enhanced visualization for breast tissues. Different experiments were conducted to evaluate the proposed framework on two different datasets that are INbreast and CDD-CESM that provides both FFDM and CESM images. The model achieved at mass detection a mean Average Precision (mAP) score of 98.69%, 81.52%, and 71.65% and mass classification accuracy of 95.65%, 97.61%, and 80% for INbreast, CE-CESM, and DM-CESM, respectively. The proposed framework showed competitive results regarding the state-of-the-art models in INbreast. It outperformed the previous work in the literature in terms of the F1-score by almost 5% for mass detection in CESM. Moreover, the experiments showed that the CESM could provide more morphological features that can be more informative, especially with the highly dense breast tissues.