What question did this study set out to answer?

This research aims to develop a statistical framework for learning in layered neural networks and to evaluate generalization capabilities.

January 1, 1990

A statistical approach to learning and generalization in layered neural networks

Key Points

This research aims to develop a statistical framework for learning in layered neural networks and to evaluate generalization capabilities.
Presents a statistical description of learning from examples in neural networks.
Derives the Gibbs distribution for networks with fixed architecture to analyze performance.
Utilizes entropy to measure the prediction distribution in terms of network performance.
Establishes a connection between minimum error and maximum likelihood criteria for neural networks.
Finds that the Gibbs distribution provides a reliable measure for the generalization ability of networks.
Shows that entropy effectively quantifies network performance in terms of prediction accuracy.

Abstract

A general statistical description of the problem of learning from examples is presented. Learning in layered networks is posed as a search in the network parameter space for a network that minimizes an additive error function of a statistically independent examples. By imposing the equivalence of the minimum error and the maximum likelihood criteria for training the network, the Gibbs distribution on the ensemble of networks with a fixed architecture is derived. The probability of correct prediction of a novel example can be expressed using the ensemble, serving as a measure to the network's generalization ability. The entropy of the prediction distribution is shown to be a consistent measure of the network's performance. The proposed formalism is applied to the problems of selecting an optimal architecture and the prediction of learning curves.>

Bookmark

A statistical approach to learning and generalization in layered neural networks

Key Points

Abstract

Cite This Study

Also Consider

Also Consider