Why the study?
Although artificial intelligence can classify electrocardiograms with high discrimination, whether model probabilities, decisions, and explanations remain reliable under technical signal failures is unclear.
Does an integrated reliability-aware evaluation framework (incorporating isotonic calibration, selective prediction, and a missing-lead input gate) improve probability reliability and reduce expected decision harm for deep-learning multilabel ECG classification under signal perturbations?
Observational (n=21,388)
Does an integrated reliability-aware evaluation framework (incorporating isotonic calibration, selective prediction, and a missing-lead input gate) improve probability reliability and reduce expected decision harm for deep-learning multilabel ECG classification under signal perturbations?
This study demonstrates that evaluating AI-based ECG classifiers with a multi-layered reliability framework—incorporating isotonic calibration, selective abstention, and input quality gating—exposes vulnerabilities to lead dropout and reduces expected decision harm that standard discrimination metrics alone fail to capture.
Artificial intelligence can classify electrocardiograms, but high discrimination alone does not show whether probabilities, decisions, and explanations remain reliable under technical signal failures. This study evaluates an integrated reliability-aware framework for multi-label electrocardiogram classification on PTB-XL with five diagnostic superclasses: normal, myocardial infarction, ST and T wave change, conduction disturbance, and hypertrophy. Patient-disjoint folds 1–8, 9, and 10 were used for training, validation, and held-out testing, respectively. XGBoost and three one-dimensional deep-learning architectures were compared, and ResNet1D was selected by validation macro area under the receiver operating characteristic curve. On the held-out test fold, ResNet1D achieved a macro area under the receiver operating characteristic curve of 0.9110, macro area under the precision-recall curve of 0.7746, and macro F1 score of 0.7051. Isotonic calibration, fitted on validation predictions only, produced a held-out macro Brier score of 0.08523 and adaptive calibration error of 0.00053. Selective prediction showed a coverage-risk trade-off: higher coverage reduced abstention but increased accepted-case risk and expected harm. Robustness tests covered Gaussian noise, baseline wander, amplitude scaling, lead dropout, and partial waveform failures; grouped limb- and precordial-lead dropout caused the largest discrimination losses. A complete-missing-lead input gate rejected zeroed or flat leads and reduced expected harm under complete lead loss, but did not detect the tested partial waveform failures. Attribution stability was high under Gaussian noise and lower under lead dropout. External analysis on Georgia and Chapman-Shaoxing data was limited to a binary abnormal-versus-normal sanity check and showed source-dependent calibration transfer. The contribution is a reproducible reliability evaluation framework, not a new architecture, full external validation, or a clinical deployment claim.
No takes yet. Share an insight, caveat, or question.
Konyspayev et al. (2026) conducted an observational in Multilabel electrocardiogram classification across five diagnostic superclasses (normal, myocardial infarction, ST and T wave change, conduction disturbance, and hypertrophy) (n=21,388). ResNet1D with reliability-aware evaluation framework (isotonic calibration, selective prediction, and complete-missing-lead input gate) vs. Temporal convolutional network (TCN), InceptionTime, and XGBoost was evaluated on Held-out test fold macro area under the receiver operating characteristic curve (AUROC) comparing ResNet1D vs Temporal Convolutional Network (MD 0.000613, 95% CI -0.003160 to 0.004178).
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: