Key result
Post-hoc recalibration improves global but not local reliability in medical time-series deep learning models.
Why the study?
Deep learning models on wearable photoplethysmography risk poor performance in practice, making reliable uncertainty quantification essential for clinical trust.
When implementing deep learning for physiological monitoring, UQ techniques must be evaluated for small-scale and local reliability rather than global calibration alone to safely inform clinical decision-making.
Local UQ reliability must be verified before clinical use of DL time-series models; leaves open optimal techniques by metric and scale.
In principle, deep learning models trained on medical time-series, including wearable photoplethysmography sensor data, can provide a means to continuously monitor physiological parameters outside of clinical settings. However, there is considerable risk of poor performance when deployed in practical measurement scenarios leading to negative patient outcomes. Reliable uncertainties accompanying predictions can provide guidance to clinicians in their interpretation of the trustworthiness of model outputs. It is therefore of interest to compare the effectiveness of different approaches. Here we implement an unprecedented set of eight uncertainty quantification (UQ) techniques to models trained on two clinically relevant prediction tasks: atrial fibrillation (AF) detection (classification), and two variants of blood pressure regression. We formulate a comprehensive evaluation procedure to enable a rigorous comparison of these approaches. We observe a complex picture of uncertainty reliability across the different techniques, where the most optimal for a given task depends on the chosen expression of uncertainty, evaluation metric, and scale of reliability assessed. We find that assessing local calibration and adaptivity provides practically relevant insights about model behaviour that otherwise cannot be acquired using more commonly implemented global reliability metrics. We emphasise that criteria for evaluating UQ techniques should cater to the model's practical use case, where the use of a small number of measurements per patient places a premium on achieving small-scale reliability for the chosen expression of uncertainty, while preserving as much predictive performance as possible.
No takes yet. Share an insight, caveat, or question.
Bench et al. (2026) studied Atrial fibrillation and blood pressure estimation. Uncertainty quantification techniques (e.g., Deep Ensembles, Monte Carlo Dropout, Quantile Regression, post-hoc recalibration) vs. Comparison among different uncertainty quantification methods and base uncalibrated models was evaluated on Uncertainty reliability metrics (e.g., Expected Calibration Error, Negative Log-Likelihood) and predictive performance. The optimal uncertainty quantification technique for medical time-series deep learning models depends heavily on the evaluation metric and scale of reliability assessed, with post-hoc recalibration improving global but not necessarily local reliability.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: