Assessment of Distractor Functionality in Multiple Choice Questions: Student Generated Versus Teacher Generated Distractors

A BSTRACT Introduction: Distractors for multiple-choice questions can be obtained by various models like expert judgment, software auto-generation, or from student misconception. Students and teachers are the two sources which can be easily used to produce plausible distractors. With this understanding, we compared the distractors functionality of MCQs generated by student misconception with those formulated by teachers. Methods: Interns were divided into 3 cohorts (n = 50) according to their unique student identification number. All participants received the study material 2 weeks prior. A set of 30 free response (FR) questions were prepared by 2 faculty and validity using Delphi process. The same FR questions were prepared as MCQs with 4 options (Paper A). The FR questions were administered to the 1 st cohort, and the top 3 incorrect responses for each item were documented. Using these incorrect responses as distractors, a new set of MCQs with same stem was prepared (Paper B). The 2 papers were administered to the second and third cohorts, respectively. Item analysis was done, and distractor functionality was compared between both sets. Results: The reliability coefficient was 0.4 and 0.81 in Paper A and Paper B, respectively. The mean difficulty index was 0.51 ± 0.3 and 0.41 ± 0.2 (Cohen’s d = 0.39), and discrimination index was 0.25 ± 0.3 and 0.36 ± 0.3 (Cohen’s d = 0.33), respectively. Items with distractor efficiency of 100%, 66.6%, 33.3%, and 0% were 13, 10, 6, 1 and 20, 8, 1, 1, respectively. The no of functional distractors per item was 1.6 and 2.5 in Paper A and Paper B. Paper B had a significant positive correlation between the discrimination index and the number of FDs (r = 0.5068; P < 0.0004). Conclusion: Distractors generated from student misconception is superior in functionality compared with teacher generated, thereby increasing the validity and discriminatory function of the test. This study confirms that incorrect answers from FR assessment can be used as a source of plausible distractors.