Three-Option Multiple-Choice Questions Retain Discrimination and Reliability: A Prospective Randomized Crossover Study in Pharmacy Education

OBJECTIVE: To prospectively compare the psychometric performance of 3-option versus 4-option multiple-choice question (MCQ) examinations and evaluate differences in item difficulty, item discrimination, examination reliability, examination performance, and elapsed testing time. METHODS: This randomized crossover study was conducted in 2 required Doctor of Pharmacy courses at a single school of pharmacy, including a first-year pharmacokinetics course and a third-year pharmacotherapy course. Students were assigned to alternating sequences of 3-option and 4-option MCQ examinations throughout the study period. Four-option examinations were developed first, after which 1 distractor was removed from each item to create the corresponding 3-option version. Item-level psychometric measures, examination-level outcomes, and elapsed testing time were compared between formats using paired analyses and linear mixed-effects modeling. RESULTS: Three-option MCQs were associated with students answering approximately 1 additional question correctly on a 50-item MCQ test compared with 4-option MCQs. No differences were observed in point-biserial discrimination or Kuder-Richardson 20 reliability estimates between examination formats. Course-specific analyses demonstrated differences in item difficulty within pharmacokinetics examinations but not pharmacotherapy examinations. Examinations containing 4-option MCQs also required longer testing times for students. CONCLUSION: Reducing MCQs from 4 to 3 answer options did not meaningfully compromise examination reliability or item discrimination in 2 required pharmacy courses. Although 4-option MCQs produced slightly greater difficulty and longer testing times, the practical educational benefit of routinely requiring a fourth answer option may be limited when distractors are not consistently plausible or functioning effectively.