What question did this study set out to answer?

Assess confidence calibration in modern neural networks compared to older architectures and evaluate post-processing techniques to ensure predicted probabilities match true accuracy.

June 14, 2017Open Access

On Calibration of Modern Neural Networks

View Full Paper

Ask AI

Bookmark

Authors

CGChuan GuoSnap (United States)GPGeoff PleissVector Institute YSYu SunBeihang University

Discussion

Member takes

Implication

Experimental evaluation demonstrates widespread miscalibration in modern neural networks across vision and text tasks, indicating temperature scaling effectively restores reliable confidence...

Key Points

Assess confidence calibration in modern neural networks compared to older architectures and evaluate post-processing techniques to ensure predicted probabilities match true accuracy.
Benchmarked confidence calibration across multiple modern neural network architectures on standard image and document classification datasets.
Analyzed the effects of model capacity and training regularizers, including network depth, width, weight decay, and Batch Normalization, on calibration error.
Evaluated several post-processing calibration strategies, comparing parametric methods such as temperature scaling against non-parametric alternatives.
Modern neural networks are substantially less calibrated than older networks, with increased depth, width, Batch Normalization, and reduced weight decay directly linked to overconfident predictions.
Temperature scaling consistently outperforms more complex calibration approaches across vision and NLP benchmarks, recovering accurate probability estimates with minimal computational cost.

Cite This Study

Guo et al. (2017) studied this question.

synapsesocial.com/papers/6a0ea5c91c5e2d2319f9aa58 https://doi.org/10.48550/arxiv.1706.04599

View Full Paper

Ask AI

Bookmark

Implication

Experimental evaluation demonstrates widespread miscalibration in modern neural networks across vision and text tasks, indicating temperature scaling effectively restores reliable confidence...

On Calibration of Modern Neural Networks

Discussion

Member takes

Implication

Key Points

Cite This Study

On Calibration of Modern Neural Networks

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider