Using weight decay to optimize the generalization ability of a perceptron

Weight decay was proposed to reduce overfitting which often appears in the generalization tasks of artificial neural nets. Here weight decay is applied to a well defined model system based on a single layer perceptron, which exhibits strong overfitting. Since we know for this system the optimal nonoverfitting solution we can compare the effect of the weight decay with this solution. A strategy to find the optimal weight decay strength, which leads to the optimal solution for any number of examples, is proposed.