Ensemble-based uncertainty quantification methods yielded more robust uncertainty estimations than single network or stochastic methods for multi-label ECG classification.
Ensemble-based uncertainty quantification methods improve the robustness and reliability of AI models for automated multi-label ECG classification, particularly for out-of-distribution detection and active learning.
Absolute Event Rate: 0.903% vs 0.896%
Artificial Intelligence (AI) use in automated Electrocardiogram (ECG) classification has continuously attracted the research community’s interest, motivated by their promising results. Despite their great promise, limited attention has been paid to the robustness of their results, which is a key element for their implementation in clinical practice. Uncertainty Quantification (UQ) is a critical for trustworthy and reliable AI, particularly in safety-critical domains such as medicine. Estimating uncertainty in Machine Learning (ML) model predictions has been extensively used for Out-of-Distribution (OOD) detection under single-label tasks. However, the use of UQ methods in multi-label classification remains underexplored. This study goes beyond developing highly accurate models comparing five uncertainty quantification methods using the same Deep Neural Network (DNN) architecture across various validation scenarios, including internal and external validation as well as OOD detection, taking multi-label ECG classification as the example domain. We show the importance of external validation and its impact on classification performance, uncertainty estimates quality, and calibration. Ensemble-based methods yield more robust uncertainty estimations than single network or stochastic methods. Although current methods still have limitations in accurately quantifying uncertainty, particularly in the case of dataset shift, incorporating uncertainty estimates with a classification with a rejection option improves the ability to detect such changes. Moreover, we show that using uncertainty estimates as a criterion for sample selection in active learning setting results in greater improvements in classification performance compared to random sampling.
Barandas et al. (Tue,) conducted a other in Cardiac pathologies (ECG classification) (n=31,386). Ensemble-based uncertainty quantification methods vs. Single network or stochastic methods was evaluated on Area Under the Receiver Operating Characteristic (AUROC) for internal validation. Ensemble-based uncertainty quantification methods yielded more robust uncertainty estimations than single network or stochastic methods for multi-label ECG classification.