Advancing breast cancer detection in ultrasound images using a novel hybrid ensemble deep learning model

Breast cancer remains a leading cause of mortality among women globally, emphasizing the critical need for prompt and accurate detection to improve patient outcomes. This study introduces an innovative hybrid model combining ultrasound image enhancement techniques with advanced machine learning for rapid and more accurate breast cancer prognosis. The proposed model integrates Contrast Limited Adaptive Histogram Equalization (CLAHE) for image quality improvement with an Ensemble Deep Random Vector Functional Link Neural Network (edRVFL) for classification. Utilizing a dataset of 4103 high-resolution ultrasound images from the Dunya Women's Cancer Center in Palestine, categorized into normal, benign, and malignant groups, the model was trained and evaluated using a 25-fold cross-validation approach. Results demonstrate higher performance of the hybrid model compared to traditional machine learning algorithms, achieving accuracies of 96 % for benign and 98 % for malignant cases after CLAHE enhancement. To further improve lesion detection and segmentation, a new method combining YOLOv5 object detection with the MedSAM foundation model was developed, achieving a Dice Similarity Coefficient of 0.988 after CLAHE enhancement. Validation in a clinical setting on 850 cases showed promising results, with 91.4 % ± 0.021 accuracy for benign and 84 % ± 0.024 for malignant predictions compared to histopathology. The model's high accuracy and interpretability, supported by Grad-CAM analysis, demonstrate its potential for integration into clinical practice. This study advances the application of machine learning in breast cancer detection from ultrasound images, presenting a valuable tool for enabling early detection and improving prognosis for breast cancer patients. • Hybrid model combines CLAHE with edRVFL for improved breast cancer classification. • CLAHE enhancement improves ultrasound image quality and model performance. • edRVFL outperforms other ML models, achieving 98 % accuracy for malignant cases. • Novel segmentation method integrates YOLOv5 and MedSAM for precise delineation. • Grad-CAM analysis provides visual explanations, enhancing model interpretability.