A Psychometric Comparison of Faculty-Authored and Large Language Model-Generated Multiple-Choice Questions in Endodontics

= 0.000918), with 13 (43.3%) classified as easy, and had a high rate of non-functional distractors (NFDs). While five (16.7%) of the LLM questions had 100% NFDs (where all distractors failed to function), the faculty cohort had no such items. A concurrency analysis revealed that three (10.0%) of the faculty items simultaneously satisfied all ideal psychometric benchmarks, compared to only one (3.3%) of the LLM items. Conclusions Although next-generation LLMs can produce stylistically authentic clinical vignettes that closely mimic human-authored writing, their raw, uncurated outputs exhibit significant psychometric limitations, particularly regarding distractor plausibility and item discrimination. Fully autonomous AI item generation is not yet viable for high-stakes assessments. A hybrid "human-in-the-loop" model, where LLMs are leveraged for rapid preliminary drafting and experienced educators perform targeted distractor refinement, can safeguard assessment validity while substantially reducing the administrative burden on dental faculty.