What question did this study set out to answer?

To develop a framework that enhances the prediction of breast cancer recurrence using distributed processing and advanced machine learning techniques.

April 6, 2026Scientific Reports0 citationsOpen Access

A distributed fusion framework for breast cancer recurrence prediction using MapReduce

PSPrachi Damodhar ShahareMahaveer Academy of Technology and Science University AMAbha MahalwarMahaveer Academy of Technology and Science University ASAniket K. ShahadeSymbiosis International University

Key Points

To develop a framework that enhances the prediction of breast cancer recurrence using distributed processing and advanced machine learning techniques.
Proposed a MapReduce-aligned hybrid framework using Spark-based Gradient Boosted Trees and a denoising autoencoder.
Utilized calibrated XGBoost for performance evaluation on two benchmark datasets: SEER and Wisconsin.
Compared framework performance in terms of ROC-AUC and other metrics across heterogeneous datasets.
Achieved an ROC-AUC of 0.9954 and MCC ≥ 0.981 on the Wisconsin dataset with calibrated XGBoost.
Demonstrated improved recall on the SEER dataset, highlighting better performance for minority-class prediction.
Showed that the fusion framework consistently outperforms strong baseline frameworks, indicating scalability and efficiency for medical records.

Abstract

Breast cancer recurrence remains a major clinical challenge, significantly influencing long-term survival and treatment planning. Accurate early prediction is hindered by heterogeneous clinical factors, imbalanced datasets, and the distributed nature of medical records stored across hospitals, registries, and laboratories. To address these challenges, this study proposes a MapReduce-aligned hybrid framework that combine with distributed Spark-based Gradient Boosted Trees, denoising autoencoder (AE)–derived latent representations, calibrated XGBoost, and deep tabular framework (FT-Transformer and TabTransformer). The framework is designed to operate efficiently on heterogeneous, large-scale datasets while preserving data locality. Two benchmark datasets; the SEER breast cancer recurrence cohort and the Wisconsin Diagnostic Breast Cancer dataset were used to evaluate framework performance across clinical data. Experimental results show that the proposed calibrated XGBoost and AE-augmented fusion frameworks obtained superior discrimination, calibration with the Wisconsin dataset reaching ROC-AUC values of 0.9954 and MCC ≥ 0.981. On the SEER dataset, characterized by high heterogeneity and sparse recurrence signals, the fusion framework attained improved recall, while calibrated XGBoost offered the best overall balance between precision and stability. The findings demonstrate that combining tree-based embedded feature selection, latent AE compression, and transformer-based contextual frameworking yields consistent performance gains. Moreover, the Spark-GBT integration ensures scalability and suitability for multi-institutional environments where data centralization is restricted. The experimental results show that the proposed fusion framework provides competitive performance compared to strong baseline frameworks such as calibrated XGBoost, and improved recall and robustness for minority-class recurrence prediction. The results indicate that fusion learning improves sensitivity and framework stability, whereas calibrated XGBoost provides the strongest overall discrimination performance. The proposed framework presents a reliable, scalable, and clinically meaningful solution for individualized recurrence-risk prediction.

问 AI

Bookmark

View Full Paper