Development of Standard Anchors for Scoring Generic Past‐Behaviour Questions in Structured Interviews

A structured interviewing system was developed which included an item bank of pre‐written past‐behaviour questions. Scoring anchors for each of 168 questions in the item bank were created from analysis of protocols from individual assessments. A concurrent validation study was conducted, focusing on the job of highway patrol sergeant. Job knowledge experts used a job analysis tool to select a subset of twelve questions from the item bank. Panel interviews were conducted with 32 highway patrol sergeants as subjects. Average inter‐rater reliability was 0.65 and interview validity was 0.42 for group consensus scores and 0.44 for panel average scores. Discussion focused on practical issues affecting the interviewing system.