Question-Order Effects on Presidential Popularity

SOCIAL scientists are becoming increasingly sensitive to the poten-tial contamination of survey data by response effects. This paper explores a type of response problem about which relatively little is known: the effect of question order (see especially Sudman and Brad-burn, 1974). The paper centers on a question that has been asked in Gallup Polls for more than 40 years: Do you approve or disapprove of the way that is handling his job as President? A question-order effect is said to operate when responses are influenced by the questions placement within a survey. Most re-search on order effects pertains to the situation in which peoples responses to one item in a survey are consciously or unconsciously influenced by their responses to a previous item; had the order of the items been reversed, or had the first one not been asked, their responses to the later question would have been different. This can happen, for example, as the result of what Bradburn and Mason Abstract A methodological experiment is conducted on the same question that is used virtually every month in the Gallup Poll to measure presidential popularity. The point of the experiment is to determine whether presidential popularity is affected by the placement of the question within the survey, in response to a recent charge that alternations in the order in which this question has been asked invalidate time-series analyses of presidential popularity. The primary finding is that the order in which the question is presented does not significantly affect the direction of response (the balance between approval and disapproval), but it does affect opinionation (the willingness to make a directional response, whether positive or negative). This effect is found to be particularly pronounced for less educated respondents. The implications of these find-ings for time series analysis of presidential popularity are spelled out.