The Random Effects in Multilevel Models: Getting Them Wrong and Getting Them Right

Many surveys of respondents from multiple countries or subnational regions have now been fielded on multiple occasions. Social scientists are regularly using multilevel models to analyse the data generated by such surveys, investigating variation across both space and time. We show, however, that such models are usually specified erroneously. They typically omit one or more relevant random effects, thereby ignoring important clustering in the data, which leads to downward biases in the standard errors. These biases occur even if the fixed effects are specified correctly; if the fixed effects are incorrect, erroneous specification of the random effects worsens biases in the coefficients. We illustrate these problems using Monte Carlo simulations and two empirical examples. Our recommendation to researchers fitting multilevel models to comparative longitudinal survey data is to include random effects at all potentially relevant levels, thereby avoiding any mismatch between the random and fixed parts of their models.