Large language models in undergraduate medical education - student use, perceptions, and outcomes: a systematic review (Preprint)

BACKGROUND Large language models (LLMs), such as ChatGPT, are increasingly being used in medical education. Medical students use them for purposes such as supporting their self-directed learning. Educators are increasingly using them to support teaching, for example in simulating patient conversations. Adoption is generally ahead of institutional endorsement or training. OBJECTIVE This review seeks to better understand how and why medical students use LLMs and their educational impact. METHODS A systematic review was conducted, following PRISMA guidelines, of studies that involved medical students using LLMs and reported students’ use, perceptions or educational outcomes. Original peer-reviewed research available in English was included between November 2022 and July 2025. Two reviewers independently screened papers, extracted data and appraised quality using CASP. Data were synthesised using a narrative synthesis framework. RESULTS 54 studies were included, including cross-sectional and interventional studies. Awareness of LLMs was very high amongst medical students, while use was more variable. Most students used ChatGPT. Students report LLMs provided benefits to learning efficiency and more personalised learning, though this did not necessarily translate to better knowledge retention. Interventional studies using LLMs to simulate patient interactions, and provide personalised feedback, improved students’ performance and confidence. Students had concerns around accuracy, data, and overreliance which coexisted with use. Verification of outputs was often low but correlated with prior understanding of AI. Students were supportive of more formal training on AI. CONCLUSIONS LLMs can be useful in supporting students’ learning and are being adopted by medical students. They can provide benefits as tools for low-stakes practice of communication skills and in providing feedback. Many students have concerns around accuracy but lack the skills to critically appraise LLM outputs. Institutions should provide formal training on LLM use to develop verification habits, model good LLM-use, and prepare students for AI they will encounter in clinical practice. CLINICALTRIAL Badakhshan U, Ahadi N, Athavale R. Large language model use among medical students in medical education: a systematic review [protocol]. Open Science Framework; 2025. doi:10.17605/OSF.IO/YMCW8.