Answers are grounded in this paper and cite their sources.
A technique for constructing neural network architectures with better ability to generalize is presented under the name Ockham's Razor: several networks are trained and then pruned by removing connections one by one and retraining. The networks which achieve fewest connections generalize best. The method is tested on a classification of bit strings (the contiguity problem): the optimal architecture emerges, resulting in perfect generalization. The internal representation of the network changes substantially during the retraining, and this distinguishes the method from previous pruning studies.
No takes yet. Share an insight, caveat, or question.
Hans Henrik Thodberg (1991) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: