Setting Performance Standards Through Two-Stage Judgmental Policy Capturing

A performance-standard-setting procedure termed judgmental policy capturing (JPC), and its application, are described. The JPC procedure was examined and evaluated as a possible means of establishing performance standards for classroom teachers who seek certification from the National Board for Professional Teaching Standards. The study demonstrated the feasibility of using the method for setting performance standards on assessments composed of complex, multiply scored performance-based exercises. The results of the study suggested that (a) the JPC procedures and associated instructions were clear to panelists, (b) panelists could provide judgments in response to as many as 200 five-dimensional profiles in a reasonably short time without undue fatigue, and (c) most panelists could achieve high levels of consistency in their responses to profiles, as reflected in coefficients of multiple determination between 0.55 and 0.83.