What question did this study set out to answer?

To develop an efficient algorithm that finds flat minima in neural network error functions to minimize model complexity and improve generalization.

January 1, 1997Neural Computation

Flat Minima

View Full Paper

Ask AI

Bookmark

Authors

SHSepp HochreiterLinköping University JSJürgen SchmidhuberUniversity of Applied Sciences and Arts of Southern Switzerland

Discussion

Member takes

Implication

Computational modeling demonstrates that searching for flat minima reduces overfitting in neural networks, indicating that broad low-error weight regions improve generalization.

Key Points

To develop an efficient algorithm that finds flat minima in neural network error functions to minimize model complexity and improve generalization.
Formulated a Minimum Description Length and Bayesian framework using a Gibbs algorithm variant to decompose generalization error into underfitting and overfitting components.
Constructed an optimization algorithm utilizing second-order derivatives while preserving standard backpropagation computational complexity.
Evaluated the search method on feedforward and recurrent architectures, benchmarking performance against backpropagation, weight decay, and optimal brain damage or surgeon techniques in stock market prediction.
The algorithm automatically and effectively pruned network units, weights, and input lines without requiring explicit Gaussian weight priors.
Flat minimum search outperformed conventional backpropagation, weight decay, and optimal brain damage/surgeon methods in financial time-series forecasting.

Cite This Study

Hochreiter et al. (1997) studied this question.

synapsesocial.com/papers/69d95f1fe6ab964fb0835aef https://doi.org/10.1162/neco.1997.9.1.1

View Full Paper

Ask AI

Bookmark

Implication

Computational modeling demonstrates that searching for flat minima reduces overfitting in neural networks, indicating that broad low-error weight regions improve generalization.

Flat Minima

Discussion

Member takes

Implication

Key Points

Cite This Study

Flat Minima

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider