Answers are grounded in this paper and cite their sources.
Following ideas of Gull, Skilling and MacKay (1992), we develop and explore a statistical-mechanics framework through which one may assign values to the parameters of a model for a 'rule' (instanced, here, by the noisy linear perceptron), on the basis of data instancing the rule. The 'evidence' which the data offers in support of a given assignment, is likened to the free energy of a system with quenched variables (the data): the most probable (MAP) assignments of parameters are those which minimize this free-energy; tracking the free-energy minimum may lead to 'phase transitions' in the preferred assignments. We explore the extent to which the MAP assignments lead to optimal performance.
No takes yet. Share an insight, caveat, or question.
Bruce et al. (1994) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: