Probabilistic Modeling in Psycholinguistics: Linguistic Comprehension and Production

It must certainly be accounted a paradox that probabilistic modeling is simultaneously one of the oldest and one of the newest areas in psycholinguistics. Much research in linguistics and psycholinguistics in the 1950s was statistical and probabilistic. But this research disappeared throughout the 60’s, 70’s, and 80’s. In a highly unscientific survey (conducted by myself) of six college textbooks and handbooks in psycholinguistics published in the last 10 years, not a single one of them mentions the word ‘probability’ in the index. This omission is astonishing when we consider that the input to language comprehension is noisy, ambiguous, and unsegmented. In order to deal with these problems, computational models of speech processing, by contrast, have had to rely on probabilistic models for over 30 years. Computational techniques for processing of text, an input medium which is much less noisy than speech, rely just as heavily on probability theory. Just to pick an arbitrary indicator, 77% of the papers in the year 2000 annual conference of the Association for Computational Linguistics relied on probabilistic models of language processing or learning. Probability theory is certainly the best normative model for solving problems of decisionmaking under uncertainty. But perhaps it is a good normative model, but a bad descriptive one. Despite the fact that probability theory was originally invented as a cognitive model of human reasoning under uncertainty, perhaps people do not use probabilistic reasoning in cognitive tasks like language production and comprehension. Perhaps human language processing is simply a non-optimal, non-rational process? In the last decade or so, there is an emerging consensus that human cognition is in fact rational, and relies on probabilistic processing. The seminal work of Anderson (1990) gave Bayesian underpinnings to cognitive models of memory, categorization, and causation. Probabilistic models have cropped up in many areas of cognition; one area in which there are a number of recent probabilistic models is categorization (Rehder 1999; Glymour and Cheng 1998; Tenenbaum 2000; Tenenbaum and Griffiths 2001b; Tenenbaum and Griffiths 2001a), Probabilistic models are also now finally being applied in psycholinguistics, drawing from early Bayesian-esque precursers in perception such as the Luce (1959) choice rule and the work of Massaro. What does it mean to claim that human language processing is probabilistic? This claim has implications for language comprehension, production and learning.