Web‐based and Paper‐and‐pencil Testing of Applicants in a Proctored Setting: Are Personality, Biodata, and Situational Judgment Tests Comparable?

This quasi‐experimental study compares the equivalence of proctored Web‐based tests to paper‐and‐pencil tests in a selection setting. The predictor battery was composed of measures of Conscientiousness, Agreeableness, Emotional Stability, a biodata form, and a situational judgment test. Three samples were administered the same test items, but differed with respect to context and test format: 425 incumbents completed a paper‐and‐pencil version, 2,544 applicants completed a paper‐and‐pencil version, and 2,356 applicants completed an identical Web‐based version (although these individuals were not randomly assigned to test format, the job and organization were the same for all samples). The results found effects for both context and test format. Relative to the applicants completing the paper‐and‐pencil measures, the Web‐based measures showed (a) better distributional properties, (b) lower means, (c) more variance, (d) higher internal consistency reliabilities, and (e) stronger intercorrelations. Although the applicant context resulted in higher mean responses than the incumbent context, this difference was about twice as large for the paper‐and‐pencil test than for the Web‐based test. All such differences were greater for personality measures than for biodata and situational judgment tests. Overall, these results suggest that proctored Web‐based testing has some positive benefits relative to paper‐and‐pencil measures, and we identify several implications of these findings for research and practice.