The introduction of the Medical Licensing Assessment (MLA) requires UK medical graduates to pass two 100-item single best answer (SBA) question papers. Access to high-quality question banks is often restricted by expensive paywalls, evolving guidelines, and institutional safeguarding of questions for future assessments. This limits students' ability to practice exam-style questions and develop their clinical reasoning. During this study, two medical students used ChatGPT to produce a bank of 100 SBA questions which were representative of those used in the MLA. Questions were analysed and revised by a student and a member of a university assessment team. A critical review of ChatGPT’s benefits was conducted to determine the scope for application in both self-directed learning and formal medical school assessment. ChatGPT successfully formulates clinical vignettes and accompanying SBA questions in less than 50% of the time taken to generate a comparable question, when written by a clinical educator. It is, however, not yet reliable enough to be used without expert supervision or direct correlation with current medical guidelines. Both medical students and clinical educators will benefit from ChatGPT’s ability to rapidly compose SBA questions that emulate those used in medical school exams, provided they remain conscious of the risk of false clinical recommendations. The acquisition and implementation of our question bank by a leading commercial medical education provider confirms the value of ChatGPT generated SBA questions as a resource for medical students preparing for the UKMLA.
McGrane et al. (Tue,) studied this question.