Generalization and Connectionist Language Learning

The performance of any learning system may be assessed by its ability to generalize from past experience to novel stimuli. Hadley (this issue) points out that in much connectionist research, this ability has not been viewed in a sophisticated way. Typically, the \-set " consists of items which do not occur in the training set � but no attention is paid to the degree of novelty of test items relative to training items. Hadley's arguments challenge connectionists to go beyond using training and test sets which arechosen according to convenience, and to carefully construct materials which allow the degree of generalization to be investigated in more detail. We hope that this challenge will encourage more sophisticated connectionist approaches to generalization and language learning. Hadley de nes di erent degrees to which a language learning system can generalize from experience, what he calls di erent degrees of systematicity. In this paper we discuss and attempt to build on Hadley's account, providing more formal and precise de nitions of these varieties of generalization. These generalizations aim to capture the cases that Hadley discusses, but also extend to other examples. We then report some connectionist simulations using simple recurrent neural networks which we assess in the light of the revised de nitions. 0This researchwas made possible by a McDonnell PostdoctoralFellowship to MHC. NC was partially supported by agrantfrom the Joint Councils Initiative in Cognitive Science/HCI, grant no. SPG 9029590. We would like thank Bob Hadley and Dave Chalmers for commenting on an earlier draft of this paper.