On a Probability Mechanism to Attain an Economic Balance Between the Resultant Error of Response and the Bias of Nonresponse

Abstract The author postulates a probability mechanism for the simultaneous production of the bias of nonresponse and for the variance of response. The nonresponse arises from a graded series of classes of the members of the universe to be sampled. The classes range from an impregnable core of no possible response, on up to a class of complete response. Nonresponse arises from two sources, not at home, and refusal. Refusals are of two kinds, permanent and temporary. The variation in the amount of time spent at home, and the variation in the firmness of the temporary refusal, produce the graded series of classes. The bias of nonresponse arises from the variation of any characteristic from one class to another. The variance of response arises from the variation of any characteristics from one member to another within a single class, and from the random variation in the number of responses therefrom. An increase in the size of the initial sample or a more efficient method of selection will decrease the variance of response, but will have no effect on the bias of nonresponse. Successive recalls, on the other hand, decrease the bias of response, and are more effective than an increase in the size of the sample or a more efficient method of selection in decreasing the root-mean-square error which arises from both nonresponse and from the variation of response. The results show that without recalls, it is hazardous to put any confidence in the result, no matter how big the sample, even when the variation in the measured characteristic is only two-fold from the class of lowest response to the class of highest response. With the levels of response assumed here (taken from average urban experience), and with an estimate formed by summing up the initial call and the recalls, the first two recalls effect together about a 50% reduction in the initial bias of nonresponse. Further recalls continue to be productive. In fact, with this method of estimation, each recall added to a sampling plan, even to six recalls, actually increases the amount of information obtained for each dollar expended on interviewing. Even with three recalls, and with only a two-fold variation from the class of lowest response to the class of highest response, an initial sample bigger than the equivalent of from 300 to 500 binomial cases in any one subclass is ineffective and uneconomical. The apparent precision of a bigger sample is a delusion, as with bigger samples the bias of nonresponse will eclipse the error of sampling unless there are 4, 5, or more recalls. An attempted “complete count” is no exception and often represents an extreme waste of effort. For high accuracy, a plan that uses the ordinary method of estimation by combining the initial attempt and the recalls must support 4, 5, or 6 recalls, along with an initial sample equivalent to from 800 to 1500 binomial cases. For any proposed survey, calculations based on rough advance estimates of the constants that appear in the formulas will predict to a useful degree of approximation the biases and the variances to be expected from various types of plans. Figures on costs will then point out which plan is most economical, of those that are possible, for the attainment of a prescribed accuracy. Where extremely high accuracy is required, the Politz plan with 2000 or more binomial cases becomes competitive in cost with a survey that depends on recalls. In any case, the Politz plan has the advantage of speed and of being able to produce results under circumstances wherein recalls are impossible (for example, listening to a radio program). The proposed mechanism provides a theory of bias to supplement the theory of sampling. It indicates the possibility of new and more efficient methods of estimation than the simple combination of the initial attempt and the recalls, as it will provide a rational basis for extracting more information from the recalls. It will also point out, for any particular method of estimation, what empirical information will be helpful in the planning of the efficient allocation of effort amongst the initial sample and the recalls.