Abstract We studied the effects of the variable biases of interviewers on responses to a variety of items, mostly attitudinal, included in two surveys. Not satisfied with “testing statistical significance,” we measured this interviewer variance as a component of the total variance per respondent; thus s 2 =s a 2 + s b 2, where s a 2 is the “between interviewer” and s b 2 the “within interviewer” component; ρ = s a 2/s 2 is the proportion of the interviewer effect. The contribution of the interviewer variance to total survey errors is examined for its implications for the planning and interpretation of survey statistics. Among these we included subclass means and comparisons among pairs of subclasses. The data yielded important results: (1) We can obtain responses with low or moderate interviewer variance on highly “ambiguous” and “critical” attitudinal questions. The range of ρ's was mostly 0 to .07 in the first study, 0 to .05 in the second and their average is about .01 or .02; these effects are not generally higher than for most items in a good Census. (2) Nevertheless these seemingly small ρ's, combined with moderate workloads, can often increase the variance by factors as high as two or three, because the ratio of increase is where is the average number of interviews per interviewer. (3) The interviewer effects on subclass means were shown to be smaller in accord with where is average number of subclass members per interviewer. (4) In the comparisons of subclass means, the interviewer effects tend to zero, in accord with an additive model of the effects.
Leslie Kish (1962) studied this question.