Answers are grounded in this paper and cite their sources.
Single Best Answer questions are widely used in medical and dental education but require substantial faculty time to develop. Automated approaches have been proposed to support assessment item development, yet evidence evaluating structured systems integrated into real assessment workflows remains limited. This study evaluated a structured platform-assisted workflow designed to support faculty in drafting single-best-answer questions from teaching materials. This prospective paired comparison study was conducted at a single dental college. Eight experienced faculty members generated assessment items using two workflows: manual authoring and a custom-built digital platform. The platform utilized the Google Gemini 2.5 Pro Application Programming Interface (API) to generate drafts within a standardized template, which were then mandatorily reviewed, edited, and finalized by faculty. To control for order effects and cognitive priming, the sequence of item generation was counterbalanced. For matched learning objectives, 300 distinct questions were produced (150 pairs). Item-level outcomes were analyzed using linear mixed-effects models accounting for clustering within faculty authors. Recorded item-finalization time (capturing full authoring time for manual items, and post-draft review and editing time for platform-assisted items, excluding document upload, generation latency, and discarded drafts) was logged automatically. All 300 finalized assessment items were evaluated by three blinded subject-matter experts using a pilot-tested five-point Likert scale. Items were subsequently incorporated into formative assessments for undergraduate and postgraduate students to determine descriptive item-level psychometric indices (difficulty and discrimination). The mean time required to finalize a manually authored item was 6.0 min (Standard Deviation [SD] 1.2), compared with 1.0 min (SD 1.4) for platform-assisted items, representing a significant reduction in recorded finalization time ( p < 0.001). Platform-assisted items received significantly higher expert quality ratings (mean 4.7 ± 0.7) than manually authored items (mean 4.1 ± 0.8) ( p < 0.001). In descriptive psychometric analyses, platform-assisted items showed a mean difficulty index of 0.40 (96% within the prespecified 0.30–0.70 range) and a mean discrimination index of 0.24 (94% meeting the ≥ 0.20 threshold), whereas manually authored items showed a mean difficulty index of 0.46 (80% within range) and a mean discrimination index of 0.26 (81% meeting the threshold). In this single-institution exploratory study, a structured, faculty-reviewed platform-assisted workflow was associated with shorter recorded item-finalization times and higher expert ratings than the institution’s manual workflow. Platform-assisted items generally demonstrated difficulty and discrimination values within the prespecified descriptive ranges in the formative assessments studied. Because the intervention combined generative drafting, a fixed template, and faculty editing, and because psychometric equivalence was not formally evaluated, the findings should not be interpreted as demonstrating that AI independently improves item quality or that the two workflows are psychometrically equivalent. The trial was prospectively registered with the Thai Clinical Trials Registry (TCTR20260125010) on 25th Jan 2026 prior to participant recruitment.
No takes yet. Share an insight, caveat, or question.
Islam et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: