Stable rankings, unstable shortlists: A cross-algorithm stability audit of SHAP feature attributions in composition-based band gap prediction

Shapley additive explanation (SHAP) values are the standard tool in materials informatics for ranking the compositional descriptors that govern a predicted property, and the resulting shortlist of top descriptors is routinely read as physical insight, yet its reproducibility across ordinary modelling choices is rarely tested. This communication audits the stability of global SHAP rankings for composition-based prediction of the experimental band gap, using the 4604-entry matbench_expt_gap dataset, 132 Magpie descriptors, and three tree-ensemble algorithms trained under repeated five-fold cross-validation across three random seeds, giving forty-five fitted models. The complete descriptor ranking is highly reproducible within an algorithm, with mean Spearman correlations of 0.88–0.96, but the truncated top-ten shortlist that is typically reported overlaps by only 0.60–0.65 within an algorithm and 0.13–0.27 across algorithms. Only the mean p-valence electron count is among the ten most important descriptors in every run; the consensus set shrinks from five descriptors at a fifty percent agreement threshold to one at eighty percent. A correlation analysis of the full descriptor matrix shows a plausible source of this instability: the 132 nominal descriptors carry the effective information content of roughly 27–36 independent dimensions, so statistically redundant descriptors substitute for one another across resamples. Extending the protocol to a ridge-regularised linear model and to a second, unrelated property, yield strength, shows that the instability is neither specific to tree-based SHAP nor to the band gap. Single-run descriptor-importance claims are therefore fragile, a cross-algorithm, cross-resample consensus reported with its uncertainty and threshold sensitivity should be used in their place, and a minimum reporting standard is proposed.