Answers are grounded in this paper and cite their sources.
Background: The application of Artificial Intelligence (AI) in medicine has become an area of increasing interest. Large Language Models (LLMs), such as OpenAI’s ChatGPT and Google’s Gemini, have previously proven the ability to pass written board examinations across multiple specialties. However, its ability to act in the examiner role for interactive oral boards simulation remains largely unexplored, especially within plastic surgery. Therefore, this study aimed to evaluate the performance of these two prominent LLMs to accurately simulate plastic surgery oral board examinations and provide high-quality educational feedback. Methods: OpenAI’s GPT-4o and Google’s Gemini 2.5 Pro LLMs were used to simulate plastic surgery mock oral scenarios. A detailed system prompt was developed to standardize examiner behavior, providing strict guidelines on exam conduct, questioning themes, and feedback. Five standard case prompts with corresponding images were given to each LLM with topics spanning across the breadth of plastic surgery. After each simulated case, the LLM generated structured feedback for the examinee. Pilot testing in the examinee role was performed by a senior surgical trainee to produce examiner responses, including deliberate inaccuracies to test each LLMs analytical capabilities. Complete case transcripts were independently reviewed by three blinded board-certified plastic surgeons for examiner performance using a 9-item 5-point Likert scale to assess logical progression, clinical relevance, error identification, and realism. Results: Overall, both LLMs had favorable composite ratings and were equally recommended as a practice tool for trainees preparing for their oral board examinations. While the models performed similarly in areas of logical progression and adaptiveness to candidate responses, nuanced performance differences were observed. Reviewers consistently noted ChatGPT’s strengths in maintaining a realistic examiner tone and high clinical relevance, whereas Gemini was noted for its more effective probing of examinee errors and superior encounter pacing. Conclusion: This study serves as proof of concept that well-known AI platforms, such as OpenAI’s ChatGPT and Google’s Gemini, can serve as an effective adjunct for educating plastic surgery residents in preparation for oral board examinations. While both LLMs performed at a high level, there are inherent strengths and weaknesses to each model. As a result, further expanding the standardized case set could allow for more domain specific comparisons between LLMs. Thus, these findings suggests that further model-specific prompt engineering could optimize LLM performance for targeted educational tasks. As LLMs and prompt engineering continue to evolve, AI’s role as a supplemental educational tool could further expand into the development of an advanced AI-based plastic surgery residency curriculum.
No takes yet. Share an insight, caveat, or question.
Almeflehi et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: