Anything Goes: Statistical Interactions Without Substantive Theory

Interaction effects are commonly investigated in psychological research. Spurious interaction effects can occur, for instance because the outcome variable is related in a nonlinear way to an independent variable. Our study had two objectives: (1) to evaluate to what extent interaction terms in linear models can approximate non-linear phenomena, and (2) to assess how well linear models capture non-standard moderation effects. In Study 1, we created population data where the outcome variable was either monotonically or non-monotonically related to a predictor. In Study 2, we created population data sets, in which effects of one predictor on the correlation between outcome and another predictor were present. In both studies, we varied the correlation between the predictor variables, the signal to noise ratio, and the sample size. We applied linear regression models, both with and without an interaction term. We assessed the Type-1-Error rate, the sample effect size of the interaction effects, and the explained variance of the estimated model in the sample and in the population.For study 1, we found that traditional interaction terms were associated with increased Type-I error rates, but small effect sizes. For study 2, we found that if the effect is non-monotonic a traditional interaction term in a linear model does not sufficiently capture such effects.We conclude that relying solely on traditional interaction terms in linear models can be misleading. It may be beneficial to consider alternative modeling approaches or more flexible methods to better capture and understand non-linear and irregular dependency effects.