An uncertainty-informed active learning approach using pseudo-labeling with variable sample weighting achieved an AUC of 87.28% for cardiovascular risk classification using only 21 annotated samples.
Does an uncertainty-informed active learning approach using Monte Carlo dropout improve the classification of carotid ultrasound images for cardiovascular risk stratification?
An uncertainty-informed active learning approach using Monte Carlo dropout can achieve high diagnostic performance for cardiovascular risk stratification in carotid ultrasound with significantly reduced labeling costs.
The integration of model uncertainty quantification in clinical decision support systems, incorporating machine learning models, can augment the models’ reliability and robustness against domain shifts while also promoting user confidence and trust. In the present study, an uncertainty-informed active learning approach, leveraging Monte Carlo dropout for uncertainty estimation, is proposed towards the development of a deep learning model able to classify carotid ultrasound images as high-risk and low-risk for cardiovascular disease. An auxiliary dataset (CUBS) is employed for the initial model's development and fine-tuning as well as the optimization of the Monte Carlo dropout's hyperparameters. A dataset (87 B-mode ultrasound sequences) from ATTIKON hospital is subsequently utilized within the framework of active learning for model retraining based on the selection of the most informative samples according to the Monte Carlo dropout uncertainty estimation. In this context, the use of three active learning strategies is investigated, including uncertainty rank selection, pseudo-labeling for certain samples, and pseudo-labeling with variable sample weighting. The obtained results indicate that pseudo-labeling with variable sample weighting yields the best performance, achieving an AUC of 87.28% with only 21 annotated samples, which account for 30% of the total training data. Thus, this work provides evidence regarding the ability of uncertainty quantification and active learning to reduce labeling costs while maintaining model performance and enhancing the robustness and reliability of cardiovascular risk prediction models.
Ganitidis et al. (Sun,) conducted a other in Cardiovascular disease risk stratification (n=87). Uncertainty-informed active learning using Monte Carlo dropout (pseudo-labeling with variable sample weighting) vs. Other active learning strategies was evaluated on AUC for classifying carotid ultrasound images as high-risk and low-risk for cardiovascular disease. An uncertainty-informed active learning approach using pseudo-labeling with variable sample weighting achieved an AUC of 87.28% for cardiovascular risk classification using only 21 annotated samples.