Efficient, Calibrated, and Interpretable Uncertainty Quantification for Deep Learning Classification

Abstract Deep learning models often produce accurate classifiers but do not always produce outputs that are easily interpretable by decision-makers or appropriate estimates of uncertainty or confidence for the classification decision. Furthermore, for some larger architectures, existing uncertainty quantification (UQ) approaches can be computationally expensive. In this paper, we propose two new variational inference loss functions for fitting deep learning classification models and obtaining approximate Bayesian uncertainty quantification. Using text and image datasets, we demonstrate that the proposed framework is more computationally tractable than standard UQ approaches like dropout, and, compared with the competing approaches we consider, yields outputs that better reflect our prior conception of class similarity making the outputs more interpretable and easier to explain to decision-makers. We also demonstrate that the proposed framework balances producing well-calibrated class probabilities with producing posterior credible sets for the class labels that have appropriate coverage.