Generalization in a Large Committee Machine

We study generalization in a committee machine with non--overlapping receptive fields trained to implement a function of the same structure. Using the replica method, we calculate the generalization error in the limit of a large number of hidden units. For continuous weights the generalization error falls off asymptotically inversely proportional to ff, the number of training examples per weight. For binary weights we find a discontinuous transition from poor to perfect generalization followed by a wide region of metastability. Broken replica symmetry is found within this region at low temperatures. The first--order transition occurs at a lower and the metastability limit at a higher value of ff than in the simple perceptron. PACS. 87.10, 02.50, 64.60C A good part of the utility of neural networks lies in their ability to learn a function from examples. Therefore there has been a good deal of theoretical work on calculating the generalization ability of different networks, i....