Multiple‐choice Versus Free‐response: A Simulation Study

Responses to a 40‐item test were simulated for 150 examinees under free‐response and multiple‐choice formats. The simulation was replicated three times for each of 30 variations reflecting format and the extent to which examinees were (a) misinformed, (b) successful in guessing free‐response answers, and (c) able to recognize with assurance correct multiple‐choice options that they could not produce under free‐response testing. Internal consistency reliability (KR20) estimates were consistently higher for the free‐response score sets, even when the free‐response item difficulty indices were augmented to yield mean scores comparable to those from multiple‐choice testing. In addition, all test score sets were correlated with four randomly generated sets of unit‐normal measures, whose intercorrelations ranged from moderate to strong. These measures served as criteria because one of them had been used as the basic ability measure in the simulation of the test score sets. Again, the free‐response score sets yielded superior results even when tests of equal difficulty were compared. The guessing and recognition factors had little or no effect on reliability estimates or correlations with the criteria. The extent of misinformation affected only multiple‐choice score KR20′s (more misinformation—higher KR20′s). Although free‐response tests were found to be generally superior, the extent of their advantage over multiple‐choice was judged sufficiently small that other considerations might justifiably dictate format choice.