Enhancing explainability in clinical deep-learning models: Latent-space variable decoding is superior to gradient-weighted class activation mapping

Figure 1. In a rigorous comparison of 2 alternative approaches to explainability of ECG DL models, models amenable to both Grad-CAM– and LSVD-based saliency analysis had similar performance. However, Grad-CAM generated activation heat maps that were highly variable from one model input to another and had low overall reproducibility, while LSVD offered direct visualization of representative examples from groups of model inputs. In addition, between-group testing of latent-space variables during LSVD provides a method for prospective evaluation of the risk DL model overfitting. DL = deep learning; ECG = electrocardiographic; Grad-CAM = gradient-weighted class activation mapping; HCM = hypertrophic cardiomyopathy; LV = left ventricular; LSVD = latent-space variable decoding; ResNet = residual connection convolutional neural network; VAE = variational autoencoder.

Figure 2. Selected examples from ResNet model saliency analysis using Grad-CAM heat maps for the detection of LV apical aneurysm (left) and massive LV hypertrophy (right). Here, select examples of Grad-CAM heat maps (A and B) offered plausible physiologic explainability, showing clear highlighting of relevant sections of the median beat waveforms. In contrast (C and D), other examples of Grad-CAM heat maps offered ambiguous insight into model explainability, highlighting information-free segments of the median beat waveforms (either before the onset of the P wave or after the termination of the T wave) or highlighting ECG waveforms diffusely without clear focus on any particular segment. When Grad-CAM heat maps were averaged across all examples (E and F), noncritical segments (eg, the P wave) or information-free segments were preferentially highlighted. Grad-CAM = gradient-weighted class activation mapping; LV = left ventricular; ResNet = residual connection convolutional neural network.

Figure 3. Saliency analysis performed using VAE-based ECG DL models for the prediction of LV apical aneurysm (left) and massive LV hypertrophy (right). Principal component analysis of ECGs from patients with and without the HCM features of interest (A and B). Centroid ECGs are shown as opaque colored circles, while 1 standard deviation of variance is represented by the larger surrounding ovoids. Representative median beat ECG waveforms decoded from the centroid ECGs enable visualization of ECG characteristics for patients with and without the HCM features of interest (C and D). DL = deep-learning; ECG = electrocardiogram/electrocardiographic; HCM = hypertrophic cardiomyopathy; KS = Kolmogorov-Smirnov; LV = left ventricular; VAE = variational autoencoder.