Assessing breast disease with deep learning model using bimodal bi-view ultrasound images and clinical information

Breast cancer is the second leading cause of carcinoma-linked death in women. We developed a multi-modal deep-learning model (BreNet) to differentiate breast cancer from benign lesions. BreNet was constructed and trained on 10,108 images from one center and tested on 3,762 images from two centers in three steps. The diagnostic ability of BreNet was first compared with that of six radiologists; a BreNet-aided scheme was constructed to improve the diagnostic ability of the radiologists; and the diagnosis of real-world radiologists' scheme was then compared with the BreNet-aided scheme. The diagnostic performance of BreNet was superior to that of the radiologists (area under the curve AUC: 0.996 vs. 0.841). BreNet-aided scheme increased the pooled AUC of the radiologists from 0.841 to 0.934 for reviewing images, and from 0.892 to 0.934 in the real-world test. The use of BreNet significantly enhances the diagnostic ability of radiologists in the detection of breast cancer.