Stability-Driven Feature Extraction–Kolmogorov–Arnold Network-Driven Ensemble Framework for Reliable Breast Cancer Detection

Breast cancer screening is a fundamentally probabilistic diagnostic task that requires precise identification of complex imaging characteristics from diverse patient cohorts. Despite improvements in deep learning techniques, current automatic tools are typically trained on well-curated datasets and do not generalize to heterogeneous data, thereby limiting their application. This study aims to address these shortcomings by introducing a more effective and generalizable framework for breast cancer classification that focuses on the stability of features, the learning of complementary representations, and improved decision modeling. The proposed methodology incorporates stability-driven feature extraction (SDFE) with a multi-branch architecture that consists of EfficientNetV2 (Convolutional neural networks (CNNs)), EfficientFormer (Vision transformers (ViTs)), and multi-layer perceptron (MLP)-Mixer models to extract various feature representations. To improve non-linear decision boundaries, it uses a Kolmogorov–Arnold Network (KAN)-based classification head and selects the most credible prediction via an adaptive voting mechanism. This model is trained using patient-level splitting on the VinDr-Mammo dataset, evaluated using five-fold cross-validation, and subsequently externally validated on the CBIS-DDSM dataset. Experimental findings demonstrate the consistent performance of the proposed model, with accuracies of 94.5% in cross-validation, 93.3% on the VinDr-Mammo test set, and 94.6% on CBIS-DDSM, surpassing other recent state-of-the-art solutions. It demonstrates enhanced robustness and cross-dataset generalization, offering a scalable, consistent framework for breast cancer classification that supports the development of computer-aided diagnostic systems.