Variations in Mean Response Times for Questions on the Computer‐adaptive Gre® General Test: Implications for Fair Assessment

ABSTRACT In a computer adaptive test (CAT), different examinees receive different sets of questions. Questions at the same overall difficulty level and meeting roughly the same content specifications could vary substantially in the amount of time needed to answer them. If the CAT is speeded (that is, if substantial numbers of students either do not finish or must guess randomly at the end to finish), individuals who happened to get a disproportionate number of questions that took a long time to answer could be disadvantaged. The purpose of this study was to determine whether–in computer‐adaptive testing situations–the administration of a disproportionate number of questions with long, expected response times unfairly disadvantages examinees. Data from 5,957 examinees who took the computer‐delivered Graduate Record Examination (GRE ® ) quantitative measure and 14,745 examinees who took the computer‐delivered GRE analytical measure were used to investigate variation in response time in light of other factors, such as mean test score and the position of the question on the test. For both measures, substantial variation in response times was found, even for items with the same specifications and same difficulty level. But despite these differences, there was no indication that the scores of students who were administered items with long expected response times were disadvantaged.