AI-Enhanced Histopathological Breast Cancer Diagnosis Using ACO-Tuned L2-Regularized ResNet50: Toward Early Detection and Clinical Integration

The most common form of cancer, which is found in females in society, is breast cancer. The diagnosis of the disease is very complex and hence often results in the mortality of the patient. Healthcare practitioners generally use different types of medical imaging-based reports for the precise and timely detection of the disease. In this work, a custom ResNet50 model is trained using the histopathological image samples of benign and malignant cases. A hybrid optimization approach has been used to improve the model performance by resizing the image to 224 × 224 normalization and handling the class imbalance by augmenting the benign samples. By limiting the augmentation to the training set, data leakage has been prevented. In this work, searching the hyperparameter space and finding the optimal values of batch size, learning rate, and epsilon is treated as one of the most crucial parts for which ant colony optimization (ACO) has been used. During model training, the amount of regularization is adjusted in such a way that it minimizes overfitting and enhances generalization based on the gradient's behavior, which has been introduced in an adaptive L2 ridge regularization scheme. Other than fine-tuning the model, ACO-based optimization and adaptive L2 regularization are performed to enhance the generalization of the model on the histopathological data. The model has shown a promising performance of 98.86% validation accuracy, which has proven the effectiveness of the customizations made to the model during implementation. Received: 11 September 2025 | Revised: 29 April 2026 | Accepted: 14 August 2026 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The BreaKHis datasets that support the findings of this study are openly available at: https://web.inf.ufpr.br/vri/databases/breastcancer-histopathological-database-breakhis/. Author Contribution Statement Debabrata Swain: Conceptualization, Validation, Resources, Visualization, Supervision, Project administration. Mahi Shah: Conceptualization, Software, Formal analysis, Resources, Writing – original draft, Writing – review & editing, Visualization. Dev Solanki: Conceptualization, Methodology, Software, Formal analysis, Data curation, Writing – original draft, Writing – review & editing, Visualization. Harsh Shah: Conceptualization, Methodology, Investigation, Visualization. Biswaranjan Acharya: Validation, Data curation, Supervision, Project administration. Davinder Paul Singh: Validation, Investigation, Supervision, Project administration. Fernando Moreira: Validation, Writing – review & editing, Supervision, Project administration, Funding acquisition.