Faculty development on artificial intelligence improves incorporation of complex assessment questions in medical physiology

Background: Physiological content is difficult to master due to the interrelated nature of organ systems. This is compounded in medical school programs by shorter time frames for content mastery and higher stakes exams. Medical licensure exams incorporate complex clinical vignettes which may not be included in basic science course exams. This is important as the National Board of Medical Examiners (NBME) Step-1 exam shifted to pass/fail in 2022, with declining pass rates since(1). It was hypothesized that generative artificial intelligence (AI) could assist faculty in authoring Step-1 style physiology assessment questions. The incorporation of board-style formative and summative questions was implemented to align course materials with licensing. Methods: Five Medical Physiology faculty, teaching 9 content areas, voluntarily participated. Their existing practice and exam questions were coded into 1 of 3 categories: Basic Science (easiest), clinical, or NBME-style (hardest). Similarly, a practice Step-1 exam from the NBME was coded into the same 3 categories. Informed consent was received, and a pre-survey administered containing 17 Likert-style, 5 open ended, and 4 yes/no questions. Likert-style questions (scored 1-5) and open-ended questions were averaged to obtain mean ± SD. Faculty members received a 1-on-1 faculty development session encompassing basic AI use to prompting for complex multiple choice question creation. Faculty members’ respective practice questions were used for the process of prompting, review, and secondary prompting as needed. After initial examples, hands-on practice was provided. Medical physiology course assessment questions were revised by each instructor using AI, and the new questions were coded to determine improvements in implementation of board-style questions. A post-survey was administered of similar content to the pre-survey, and an interview conducted assessing perceptions of AI use for question revisions and application in coursework. Results: Pre-surveys showed high concern about Step-1 performance, but limited use of board-style questions in curriculum. Faculty overpredicted their practice and exam difficulty at 14.44% and 22.78% NBME-style questions, respectively. The research team subsequently coded 250 practice and 187 exam physiology course questions into the aforementioned categories. Of those questions, only 2.82% of practice and 6.77% of exam questions met NBME classification Vs. 91.6% in practice Step-1 exams. Post-intervention, NBME-style questions increased to 46.1% (practice) and 40.4% (exam). Faculty members indicated AI use saved time in clinical vignette authorship and few revisions were needed to ensure questions met their standards. Post-surveys indicated that AI was easy to use for question authorship and was able to write effective complex multiple-choice questions. Conclusions: Our results indicated high adoption of AI use after a formal faculty development session. Faculty perceptions of AI were positive due to ease of use and its time saving ability. AI successfully aligned course exams toward the question style of the Step-1 licensure exam which could otherwise be aversive due to the time required to author complex clinical vignettes. Future work will evaluate whether increased NBME-style questions improve Step-1 pass rates. 1. Performance Data www.usmle.org : National Board of Medical Examiners; 2025 [cited 2025 05/12/2025]. Available from: https://www.usmle.org/performance-data . This abstract was presented at the American Physiology Summit 2026 and is only available in HTML format. There is no downloadable file or PDF version. The Physiology editorial board was not involved in the peer review process.