What question did this study set out to answer?

This study aims to review and compare accuracy estimation methods, specifically cross-validation and bootstrap, for model selection.

January 1, 1995Open Access

A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection

View Full Paper

Ask AI

Bookmark

Authors

Comparison of accuracy estimation methods reveals ten-fold stratified cross-validation is optimal for model selection.

This study aims to review and compare accuracy estimation methods, specifically cross-validation and bootstrap, for model selection.
Conducted a large-scale experiment with over half a million runs of C4.5 and a Naive-Bayes algorithm.
Vary the number of folds for cross-validation and the number of bootstrap samples for the bootstrap method.
Evaluate effects on model selection using real-world datasets.
Ten-fold stratified cross-validation yielded the best results for model selection among the tested methods.
Leave-one-out cross-validation was less effective than ten-fold cross-validation despite its higher resource demands.

Ron Kohavi (1995) studied this question.

View Full Paper

Ask AI

Bookmark