Powerful Moderator Variables in Behavioral Science? Don’t Bet on Them (Version 3)

The current upsurge of interest in research replicability (and the exposure of many failures of reproducibility) has led to a much discussion about the possible role of statistical moderation (i.e., variable × variable interactions) in behavioral and social science. These interactions are so widespread and powerful, it is often argued, that we should hardly be surprised when attempts to reproduce important findings frequently lead to failure. Prior literature provides little empirical evidence about how common powerful moderation is. Using five large-scale behavioral research datasets we sought to shed light on the issue. The data reflected several thousand people engaging in a variety of behaviors over considerable periods of time, and included hundreds of demographic and psychological independent variables (IVs). For each outcome variable, we measured the interaction of every pair of IVs. Many IVs had sizable main effects on behavior, but interactions were usually very small in magnitude. While there is no doubt that interactions can occur in behavioral science contexts, the priors revealed here suggest that this should be postulated as a last, not a first resort.