The Acquisition of Anaphora by Simple Recurrent Networks

This article applies Simple Recurrent Networks (SRNs; Elman 1991 Elman, Jeffrey L. 1991. Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7: 195–225. Crossref, Web of Science ® , Google Scholar, 1993 Elman, Jeffrey L. 1993. Learning and development in neural networks: The importance of starting small. Cognition, 48: 71–99. Crossref, PubMed, Web of Science ® , Google Scholar) to the task of assigning an interpretation to reflexive and pronominal anaphora. This task demands more refined sensitivity to syntactic structure than has been previously explored. Measured quantitatively, SRNs perform quite well. However, the way in which they achieve such performance diverges in key respects from the target grammar: (i) linear N-V-reflexive/pronoun sequences affect the SRN's interpretations, even without a relevant structural relation, yielding errors unlike those made by humans; (ii) the SRN's representations distinguish sentence types, inhibiting structural generalization; (iii) the SRN's knowledge of the conditions on anaphoric dependencies fails to generalize to novel lexical items. These results have important consequences not only for the viability of SRNs as models of language learning but also for the systematicity of generalization in neural networks (Hadley 1994 Hadley, Robert F. 1994. Systematicity in connectionist language learning. Mind and Language, 9: 247–272. Crossref , Google Scholar; Marcus 1998 Marcus, Gary F. 1998. Rethinking eliminative connectionism. Cognitive Psychology, 37: 243–282. Crossref, PubMed, Web of Science ® , Google Scholar).