Response-Order Effects and the Early Gallup Split-Ballots

A meta-analysis of split-ballots conducted by the Gallup Organization in the 1930s, 1940s, and early 1950s shows that response-order effects were generally small in magnitude when averaged across a great variety of topics and questions—and as compared with many of those reported in the response-effects literature today. When analyzed by various question characteristics, the results provided some support for predictions derived from current cognitive models of response-order effects, particularly those from satisficing theory. As predicted, questions asked orally were more likely to generate a statistically significant pattern of recency effects if the response alternatives or the questions as a whole were longer rather than shorter. Other predicted patterns of primacy and recency effects failed to materialize, however, perhaps largely because of the inherent design limitations and partial confounding of question attributes in any such secondary analysis of archival survey data, but perhaps, too, because of simple chance variations. The data from these early experiments nonetheless provide a partial, albeit limited, test of rival hypotheses and explanations of response-order effects in the literature.