Scoring Alternatives for Incomplete Computerized Adaptive Tests1

The purpose of this study was to investigate an alternative method of scoring incomplete adaptive tests in the context of Educational Testing Service (ETS ® ) CAT programs. The method under study is based on a procedure originally developed for tests administered as part of the computerized Armed Services Vocational Aptitude Battery. It involves the use of regression‐based “penalty functions” to adjust scores on incomplete adaptive tests based on the number of items answered. The results of the study support the viability of the proposed method in operational settings. Cross‐validation impact analyses indicated that the regression‐based penalty functions that were developed from different CAT item pools resulted in similar levels of score adjustment for the adaptive tests included in the study. Applications of the method to real candidate data indicated that scores for candidates with incomplete tests would be slightly higher if penalty functions—rather than proportional scoring—were used to make the score adjustments.