The Analysis of Factorial Surveys

Factorial surveys constitute a specific technique for introducing experimental designs in sample surveys. Respondents are presented with descriptions (vignettes) of a constructed world in which important factors are built in experimentally. Using balanced designs well known from the multivariate experimental tradition, it is possible to build in a relatively large number of factors and levels. Within this context, the normal hypothesis is that responses are consistent on the individual level, but not totally idiosyncratic. In the analysis, it is important to determine the influence of both the vignette and the respondent variables. Analysis models for this type of data should reflect the fact that factorial surveys produce data pertaining to two distinct levels: the individual level and the vignette level. Such models are available and are generally known as multilevel analysis models. This article discusses the properties of factorial survey designs and some analysis models that address the multilevel aspects of the data. An example is presented using data on judgments on the fairness of incomes.