What question did this study set out to answer?

To examine how predictions from various supervised learning algorithms relate to true posterior probabilities and determine the effectiveness of post-processing calibration methods.

January 1, 2005

Predicting good probabilities with supervised learning

View Full Paper

Ask AI

Bookmark

Authors

ANAlexandru Niculescu-MizilNEC (United States)RCRich CaruanaMicrosoft (United States)

Discussion

Member takes

Implication

Empirical analysis reveals distinct probability distortions across supervised learning algorithms, indicating that post-hoc calibration optimizes predictions in boosted trees and margin methods.

Key Points

To examine how predictions from various supervised learning algorithms relate to true posterior probabilities and determine the effectiveness of post-processing calibration methods.
Evaluated predicted probability distributions across multiple classifier families, including maximum margin models (boosted trees, boosted stumps), Naive Bayes, neural networks, and bagged trees.
Applied and compared Platt Scaling and Isotonic Regression to correct biased probabilities across varying data sample sizes.
Maximum margin methods produced a characteristic sigmoid-shaped distortion pushing probabilities away from 0 and 1, whereas Naive Bayes pushed probabilities toward extreme values of 0 and 1.
Neural networks and bagged trees naturally output well-calibrated posterior probabilities without requiring post-processing corrections.
Following calibration with Platt Scaling or Isotonic Regression, boosted trees, random forests, and support vector machines achieved the highest overall probability prediction accuracy.

Cite This Study

Niculescu-Mizil et al. (2005) studied this question.

synapsesocial.com/papers/69df018db1b9c2de2006b09e https://doi.org/10.1145/1102351.1102430

View Full Paper

Ask AI

Bookmark

Implication

Empirical analysis reveals distinct probability distortions across supervised learning algorithms, indicating that post-hoc calibration optimizes predictions in boosted trees and margin methods.

Predicting good probabilities with supervised learning

Discussion

Member takes

Implication

Key Points

Cite This Study

Predicting good probabilities with supervised learning

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider