The Accuracy of Measurements with Probability and Nonprobability Survey Samples: Replication and Extension

Many studies in various countries have found that telephone and internet surveys of probability samples yielded data that were more accurate than internet surveys of nonprobability samples, but some authors have challenged this conclusion. This paper describes a replication and an expanded comparison of data collected in the United States, using a variety of probability and nonprobability sampling methods, using a set of 50 measures of 40 benchmark variables, larger than any used in the past, and assessing accuracy using a new metric for this literature: root mean squared error. Despite substantial drops in response rates since a prior comparison, the probability samples interviewed by telephone or the internet were the most accurate. Internet surveys of a probability sample combined with an opt-in sample were less accurate; least accurate were internet surveys of opt-in panel samples. These results were not altered by implementing poststratification using demographics.