Comparability of Paper‐and‐pencil and Computer Adaptive Test Scores on the Gre® General Test

ABSTRACT This report summarizes a study conducted to assess the comparability of paper‐and‐pencil (P&P) and computer adaptive test (CAT) scores on the GRE ® General Test. Volunteer examinees from around the country were randomly assigned to take the test in either CAT mode or P&P mode. Results indicated that for each measure (verbal, quantitative, analytical), mean scores on the CAT were higher than mean scores on P&P. It was hypothesized that CAT examinee test‐taking behavior with regard to the CAT scoring method that was in place at the time of the study (allowing student to receive a score by answering only 80% or more of the questions) may have affected CAT scores. Investigation of this hypothesis indicated that CAT examinees who did not complete their CATs obtained higher mean scores than would be predicted. However, mean scores for examinees who completed their CATs were similar to mean scores for P&P examinees. A new psychometrically‐defensible CAT scoring method, called proportional scoring, was developed in which it is to the examinees' advantage, in terms of maximizing their scores, to carefully consider and answer as many items as they can. The proportional scoring method has been implemented in the operational GRE CAT program, and it is believed that this scoring method will result in CAT scores that are comparable to P&P scores. GRE CAT scores will continue to be closely monitored.