Standard-setting plans for the NBME comprehensive Part I and Part II examinations

The National Board of Medical Examiners (NBME) has reviewed its procedure for setting pass-fail standards in conjunction with the introduction of its comprehensive Part I and Part II examinations in 1991. This report gives background information on the procedures used for the past decade to set pass-fail standards for the Part I and Part II examinations, an overview of the NBME's research on standard setting, under way since 1987, and a statement of its plans for determining pass-fail standards for these examinations. In 1981 the NBME changed from the norm-referenced standard, used since the 1950s, to a criterion-group approach to setting pass-fail standards. Although the criterion-group system resulted in more stable standards, it still meant that the standard moved whenever the performance of the reference group changed. After conducting research, surveying constituencies, and examining alternatives, the NBME has adopted a new standard-setting plan that has the following components: a content-based standard-setting procedure; determination of standards by an appropriate group; use of a fixed standard; and periodic review of standards and standard-setting procedures. This new process will produce three types of improvements: it will incorporate deliberations informed by a wide range of information, including content review; annual review of examinees' performances and pass-fail results and triennial restudy of the process will add further quality control; and a fixed standard will mean that comparable performances will be required across administrations in order to pass.