Outlier Detection and Treatment in I/O Psychology: A Survey of Researcher Beliefs and an Empirical Illustration

Extreme data points, or outliers, can have a disproportionate influence on the conclusions drawn from a set of bivariate correlational data. This paper addresses two aspects of outlier detection. The results of a survey regarding how published researchers prefer to deal with outliers are presented, and a set of 183 test validity studies is examined to document the effects of different approaches to the detection and exclusion of outliers on effect size measures. The study indicates that: (a) there is disagreement among researchers as to the appropriateness of deleting data points from a study; (b) researchers report greater use of visual examination of data than of numeric diagnostic techniques for detecting outliers; and (c) while outlier removal influenced effect size measures in individual studies, outlying data points were not found to be a substantial source of variance in a large test validity data set.