The Effect of Anchor Item Characteristics on Equivalent Cutting Scores

In competency-testing programs, score equating is often used to ensure equivalent pass/fail decisions over different forms of the same examination. When a judgmental standard-setting method (e.g., Angoff, 1971) is used to pick the pass/fail point, the designs and statistical techniques of score equating can be applied, with favorable results, to the experts' judgments rather than examinees' performances (Norcini, 1990). The purpose of this study was to determine whether anchor item sets varying in difficulty and discrimination affect the precision of cutting score equivalents generated through judge rescaling as much as equivalents produced by score equating. Analyses were based on four groups of experts (ns = 9 and 11) and two groups of examinees (ns = 250 and 1,000). Tucker equating (Gulliksen, 1950) was applied to obtain cutting score equivalents, and the expert rescaling produced similar results regardless of the characteristics of the anchor sets; rescaling was also more precise than equating. For score equating, the high-discriminating anchor sets were more precise than the low-discriminating anchor sets, especially for the hard items.