A quantitative framework for explainability assessment in medical imaging

Introduction Deep learning has achieved strong results in histopathological diagnosis, but the lack of transparency in deep neural network models limits their clinical acceptance. Explainable Artificial Intelligence (XAI) offers a principled route to address this gap by quantifying the reliability and morphological coherence of model predictions in support of clinical decision-making. Methods We present an integrated, explainability-augmented deep learning framework for binary histopathological classification on the BreaKHis dataset. The pipeline combines knowledge-distilled InceptionV3 → DenseNet-121 training with three complementary local explanation methods—Gradient-weighted Class Activation Mapping (Grad-CAM), SHapley Additive exPlanations (SHAP), and Local Interpretable Model-agnostic Explanations (LIME). Per-image outputs are aggregated at the dataset level into the ExpiScore composite interpretability index, which captures salience reliability, cross-method coherence, and morphological alignment. Additional regression and decision-tree modules build a trust meta-model that estimates the probability of prediction errors and supports automated triage of clinical cases. Two operational modes are evaluated and kept strictly separate throughout: a fully-automatic mode (CNN classification with no human intervention) and a human-in-the-loop mode (meta-model routing of high-risk cases to expert review). Results In fully-automatic mode, the distilled DenseNet-121 student achieves accuracy 0.9916, F1-score 0.9939, Area Under the receiver-operating-characteristic Curve (AUC) 0.9989, sensitivity 0.9939, and specificity 0.9866 on the 1,187-image held-out test set, with five-fold cross-validation (BreaKHis official folds) further indicating generalization stability (mean accuracy 0.8962 ± 0.0177; mean AUC 0.9598 ± 0.0094). In human-in-the-loop mode, a meta-model trained on XAI-derived features and independently evaluated on the held-out test set routes the 20% of cases with the highest predicted error probability to expert review, raising post-routing accuracy from 0.9898 to 0.9983 (+0.85 pp) and reducing false negatives by 83.3% under the assumption of correct expert resolution of all routed cases. Discussion The two regimes are not directly comparable and are reported separately throughout the paper. The fully-automatic figures characterize purely algorithmic performance; the human-inthe-loop figures characterize a clinical-decision-support workflow in which the framework prioritizes cases for expert review. Together they suggest that ExpiScore-driven routing carries discriminative predictive signal for error detection and supports a principled trade-off between predictive accuracy and interpretative transparency, providing held-out test evidence for the potential clinical utility of the proposed routing mechanism within this evaluation setting. External multi-cohort validation and formal calibration of the triage thresholds are required before clinical deployment.