Commonly Used Statistical Models in Psychology are Not Equipped to Deal with Real-World Conditions: A Simulation

Ordinary Least Square (OLS) linear models produce non-optimal parameter estimates and biased hypothesis tests when their statistical assumptions are violated. In a recent study, we mapped out the extent to which OLS models in psychology meet the assumption of normal and homoscedastic errors reporting that violations are to be expected in practice. Here we simulated the conditions typical for psychology research and evaluated the performance of OLS models, bias-corrected and accelerated bootstrap, t-bootstrap, heteroscedasticity-consistent standard errors (HCSE), MM-estimator, Design Adaptive Scale Estimator, and robust trimming methods. We varied the level of skewness and kurtosis, heteroscedasticity, sample size, sample size ratio for group designs, effect size, type and number of predictors, and model design. We found that OLS outperformed other methods only in a narrow set of conditions. M-type estimators and trimming provided more accurate parameter estimates, while bootstrapping methods and HCSE outperformed other methods in light-tailed symmetrical distributions with heteroscedasticity. We discuss the utility of applying robust methods in conditions that are common is psychology research.