What question did this study set out to answer?

Develop a memory-efficient uncertainty quantification method for deep learning that mitigates the high parameter storage overhead of Bayesian neural networks and deep ensembles.

May 30, 2021Open Access

Sparse Uncertainty Representation in Deep Learning with Inducing Weights

View Full Paper

Ask AI

Bookmark

Authors

HRHippolyt RitterUniversity College London MKMartin KuklaMicrosoft Research (United Kingdom)CZCheng ZhangShaanxi University of Science and Technology

Discussion

Member takes

Implication

Algorithm evaluation demonstrates competitive uncertainty estimation in deep neural networks with reduced memory footprints, indicating scalable Bayesian inference.

Key Points

Develop a memory-efficient uncertainty quantification method for deep learning that mitigates the high parameter storage overhead of Bayesian neural networks and deep ensembles.
Augmented layer weight matrices with a low-dimensional set of inducing weights to project uncertainty quantification into smaller parameter spaces.
Extended Matheron's conditional Gaussian sampling rule to achieve rapid weight sampling and maintain fast inference run-times.
Benchmarked predictive accuracy and uncertainty estimation performance on fully connected neural networks and ResNets.
Achieved prediction accuracy and uncertainty estimation competitive with state-of-the-art Bayesian neural networks and deep ensembles.
Reduced total parameter storage to ≤ 24.3% of the size required by a single deterministic neural network.

Cite This Study

Ritter et al. (2021) studied this question.

synapsesocial.com/papers/6a10ed9e57ab95e9a3ee301b https://doi.org/10.48550/arxiv.2105.14594

View Full Paper

Ask AI

Bookmark

Implication

Algorithm evaluation demonstrates competitive uncertainty estimation in deep neural networks with reduced memory footprints, indicating scalable Bayesian inference.

Sparse Uncertainty Representation in Deep Learning with Inducing Weights

Discussion

Member takes

Implication

Key Points

Cite This Study

Sparse Uncertainty Representation in Deep Learning with Inducing Weights

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider