Breast Cancer Detection and Classification Using Hybrid Feature Selection and DenseXtNet Approach

Breast Cancer (BC) detection and classification are critical tasks in medical diagnostics. The lives of patients can be greatly enhanced by the precise and early detection of BC. This study suggests a novel approach for detecting BC that combines deep learning models and sophisticated image processing techniques to address those shortcomings. The BC dataset was pre-processed using histogram equalization and adaptive filtering. Data augmentation was performed using cycle-consistent GANs (CycleGANs). Handcrafted features like Haralick features, Gabor filters, contour-based features, and morphological features were extracted, along with features from deep learning architecture VGG16. Then, we employed a hybrid optimization model, combining the Sparrow Search Algorithm (SSA) and Red Deer Algorithm (RDA), called Hybrid Red Deer with Sparrow optimization (HRDSO), to select the most informative subset of features. For detecting BC, we proposed a new DenseXtNet architecture by combining DenseNet and optimized ResNeXt, which is optimized using the hybrid optimization model HRDSO. The proposed model was evaluated using various performance metrics and compared with existing methods, demonstrating that its accuracy is 97.58% in BC detection. MATLAB was utilized for implementation and evaluation purposes.