Psychology Researchers' Self-Reported Knowledge of Sources of Bias in General Linear Models and How it Affects Their Analytic Practice

General Linear Models (GLM) are commonly employed in psychology research. GLMs require a number of conditions to be met in order to provide unbiased and optimal parameter estimation and hypothesis tests. These conditions include adherence to statistical assumptions, and absence of outliers and influential cases. The extent to which psychology researchers attend to these sources of bias remains under-researched. Here we present the results of a self-report survey completed by 794 psychology researchers. Participants were presented with analytic scenarios and indicated the frequency with which they attend to GLM conditions. We also surveyed researchers' theoretical understanding of GLM conditions, and the methods they use to detect and address problems. We fitted Bayesian ordinal models and used highest posterior density (HPD) intervals to compare researchers' self-reported knowledge and statistical practice across scenarios, their subject area, and their involvement in teaching research methods and statistics. We found that researchers often do not attend to violations of GLM conditions, and there are gaps in their understanding of the consequences of these violations. We found no substantial differences across psychology subject areas, and the scores of research methods instructors were comparable to those of non-teaching faculty. We discuss the implications of these findings in the context of the reproducibility crisis and the credibility movement in psychology.