Can We Trust Concept-Based ECG Explanations? A TCAV Analysis of Confounding and Reproducibility in Atrial Fibrillation Detection

AI explanation methods are typically validated in one of two ways: by confirming that an explanation is not an artifact of the underlying data (a confound check), or by confirming that it generalizes to unseen data. Fewer studies examine whether an explanation is stable across independently trained copies of the same model. We present a validation framework combining both dimensions — confound control and model-seed reproducibility — and demonstrate it in a case study using a deep learning model trained to detect atrial fibrillation (AF) from long, continuous ECG recordings. Using concept activation vectors (CAVs), we test whether the model's predictions are associated with three clinically meaningful patterns: irregular heartbeat timing, the presence of abnormal beats, and proximity to a rhythm change. We find that a concept-prediction association can be statistically significant and survive a targeted AF-label confound control in one trained model, yet reverse direction when the same architecture is retrained from a different random initialization. Across the nine concept-depth combinations examined, three replicated consistently across five independently trained models, two were inconclusive, and four showed evidence of directional instability under the predefined criterion. These findings demonstrate, within this AFNet/LTAFDB case study, that statistical significance and confound robustness alone may not establish a stable concept-based explanation; model-seed reproducibility provides a complementary validation dimension that can reveal sensitivity to training initialization.