An Interpretable Explainable Artificial Intelligence Framework for Urinary Cancer Survival Classification using Hybrid Feature Selection and Stacked Ensembles

Introduction: Urinary cancer continues to be a public health hazard throughout the world, indicating that new classification methods for the detection in its early phase are necessary. The Surveillance, Epidemiology, and End Results (SEER) urinary cancer dataset presents a unique challenge in part due to heterogeneous data, variation in clinical features, and missing data, which affects patient outcomes. Methods: This study presents a solution to find missing data using a Random Forest Regressor for numerical data and a Random Forest Classifier for categorical data. It then describes a hybrid analysis combining Analysis of Variance (ANOVA) and Recursive Feature Elimination (RFE) for feature selection in large datasets. Urinary cancer survival classification was performed using seven traditional machine learning models, including Logistic Regression, Naive Bayes, LightGBM, XGBoost, CatBoost, Random Forest, and Decision Tree. Results: The experimental results show that the proposed method demonstrates the highest performance when compared to traditional classifiers. The proposed stacked ensemble model of base layers, which include LightGBM, CatBoost, and TabNet with a meta-layer of Logistic Regression, achieved the highest accuracy of 0.9886 and a ROC score of 0.9889. Discussion: The existing techniques have limited predictive accuracy, poor handling of complex data, and a lack of interpretability needed for clinical decisions, and the proposed stacked ensemble model successfully overcomes these limitations by utilizing a hybrid method of feature selection and ensemble learning for greater robustness. Conclusion: To promote model transparency and explainability, we implement a hybrid Shapley Additive explanations (SHAP) - Local Interpretable Model-agnostic Explanations (LIME) explainer. The results demonstrated improvement in classification accuracy and contributed to understanding model decisions. Overall, the framework was effective in predicting and analyzing urinary cancer survival.