Background: As computerized adaptive testing (CAT) becomes more prevalent, it is important to confirm that the computerized adaptive and paper‐and‐pencil (P&P) examinations offer comparable validity and statistical performance. Purpose: The purpose of this study was to investigate the validity and statistical properties of automated item selection (CAT) compared to manual item selection (P&P). Methods: A committee of specialists rated computerized adaptive tests (CATs) and P&P examinations with regard to face validity, adherence to test specifications, ordering of items, and cognitive skill distribution. The psychometric properties were compared. Results: Results indicated that the CATs and P&P examinations were comparable for face, content, and construct validity, as well as psychometric characteristics. Conclusions: Tests constructed automatically by the computer or manually for P&P can meet the criteria for validity and statistical performance. These findings generalize to any carefully developed examination program.
No takes yet. Share an insight, caveat, or question.
Lunz et al. (1996) studied this question.