An Analysis of the Accuracy of "Trial Heat" Polls During the 1992 Presidential Election

There is little systematic research on the multitude of factors that influence the accuracy of poll results. This article examines six methodological factors directly under the survey researcher's control and two factors concerning the nature of public opinion as sources of survey error. Data for this study come from 56 trial heat polls conducted during the last month of the 1992 presidential election. The most important variables influencing survey accuracy were the number of days a poll is in the field, which increased total accuracy one-half of a percentage point per day; conducting interviews only on weekdays (and thus only during evening hours), which reduced overall accuracy rates by more than 1 percentage point; and conducting a tracking poll, which increased accuracy by about 1.5 points. Sample size was not related to accuracy rates. Results also indicated that sampling frames of likely (relative to registered voters) tended to overestimate support for George Bush and underestimate support for Ross Perot, that interviewing only on weekdays led to overestimates of support for Bush, and that strict methods of defining a respondent as supporting a candidate hurt the two newcomers, Perot and Bill Clinton, more than Bush. In light of these data it is recommended that the common practice of reporting margins of based solely on sample sizes be abandoned as misleading and replaced by a more empirically justifiable measure based more on response rates. One of the favorite postelection retrospectives played after every presidential campaign is How well did the polls do in predicting levels of support for the different candidates? (see, e.g., Broder 1988; Costikyan and Kaplan 1984; Morin 1989; Richburg 1984; Roper 1983). That question seems particularly pointed after the 1993 campaign, with the added volatility caused by the Perot candidacy, and controversies surRICHARD R. LAU iS professor of political science at Rutgers-The State University. He would like to thank Janice Ballou, Gerry Pomper, Cliff Zukin, and two anonymous reviewers for helpful comments on earlier drafts of this article. Public Opinion Quarterly Volume 58:2-20 ? 1994 by the American Association for Public Opinion Research All rights reserved. 0033-362X/94/5801-0003$02.50 This content downloaded from 157.55.39.123 on Mon, 18 Jul 2016 06:00:49 UTC All use subject to http://about.jstor.org/terms An Analysis of Trial Heat Polls 3 rounding wildly fluctuating results due, by some accounts, to poorly explained midcampaign changes in methodology (Brennan 1992; Houston 1992; Morin 1992; Rosenstiel 1992; Taylor and Krane 1993; but see also Hugick, Molyneux, and Norman 1993).1 President Bush's late surge in the polls, the noticeably greater vigor of his campaigning during the last few weeks of the 1992 campaign, and the general underestimation of the magnitude of the Perot vote may have all been the result of methodological artifacts due to shifts in sampling frames (registered to likely voters) or how voters are allocated between the candidates. Add to this the overestimation of the magnitude of Clinton's victory by the early exit polls on election day (Public Perspective 1993), and we have a fairly mixed picture of polling results for the 1992 campaign. Some fluctuation in poll results is inevitable, of course, due to the well-known principles of sampling from any random distribution: the larger the sample, the smaller the sampling error that can be expected. What is much less well known is how other aspects of the survey research process, including choice of sampling frames, response rates, the instability of public opinion, and so on, affect fluctuating poll results. In fact, Buchanan (1986) estimates that observed fluctuations in poll results vary more than twice as much as would be expected by sampling error alone. Converse and Traugott (1986) do an excellent job of listing the variety of factors that could affect accuracyincluding sample size, house effects, sample composition, interviewer training, questionnaire construction, and so on-but they could report little systematic research that simultaneously considered a host of such factors. Only Crespi (1988, chap. 9) reports a multivariate analysis that considers the impact of a number of (sample size, importance of accuracy as a goal for the survey researcher, number of days until the election) and (primary or general election, whether an incumbent is running for reelection, turnout rates for the election, of victory of the winning candidate, percent undecided in the general public) factors, where endogenous refers to factors under the survey researcher's control, and exogenous refers to characteristics of the environment beyond the researcher's control. The most striking aspect of Crespi's multivariate results is that sample size-the one factor on which we base margin of calculations in reported poll results-did not significantly predict error rates independent of the other variables in the equation. This article is an attempt to contribute to this line of research. I 1. Whether the misunderstanding was the fault of the survey researchers or the media who report their results is another question. This content downloaded from 157.55.39.123 on Mon, 18 Jul 2016 06:00:49 UTC All use subject to http://about.jstor.org/terms