Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

Deep neural networks (NNs) are powerful black box predictors that have achieved impressive performance on a wide spectrum of tasks. predictive uncertainty in NNs is a challenging and yet unsolved. Bayesian NNs, which learn a distribution over weights, are currently state-of-the-art for estimating predictive uncertainty; however these significant modifications to the training procedure and are expensive compared to standard (non-Bayesian) NNs. We propose alternative to Bayesian NNs that is simple to implement, readily, requires very little hyperparameter tuning, and yields high predictive uncertainty estimates. Through a series of experiments on and regression benchmarks, we demonstrate that our method well-calibrated uncertainty estimates which are as good or better than Bayesian NNs. To assess robustness to dataset shift, we evaluate predictive uncertainty on test examples from known and unknown, and show that our method is able to express higher uncertainty out-of-distribution examples. We demonstrate the scalability of our method evaluating predictive uncertainty estimates on ImageNet.