A Student-Centered Approach Towards Implementing Large Language Models (LLMs) in Medical Education

Artificial intelligence (AI) generative text tools built on large language models (LLMs) are used by medical students, often without formal training. Research on students’ use of LLMs is limited. This study examined students’ self-reported experiences with LLMs and their expectations for the integration of LLMs in medical education. A 30-item cross-sectional email-based survey was distributed at two US medical schools. Survey results were analyzed using descriptive statistics, with non-parametric tests, and ordinal logistic regression was used to assess patterns of LLM engagement. Respondents were stratified into low, moderate, and high-LLM usage groups using k-means clustering, and heatmaps of user frequency were generated. A total of 103 students, spanning all levels of training within medical school, completed the survey. High- versus low-usage users reported greater LLM knowledge (3.46±0.82 vs 2.40±0.91, p = 0.0004), more confidence judging LLM HIPAA-compliance (p = 0.016), stronger agreement that LLMs improve medical education (4.14±0.79 vs. 2.16±1.19, p < 0.0001), and more LLM clinical application predictions (p = 0.0001). Usage (p = 0.007), not institution (p = 0.714) predicted educator-driven LLM engagement. LLMs were most helpful for fact-finding, literature summarization, and differential diagnosis and least helpful for flashcards. High-usage users noted inaccuracies more frequently (p <0.0001). Students anticipated future roles in documentation, administrative tasks, literature review, and patient education. Most (86.4%) endorsed formal training, emphasizing critical thinking and ethical/legal skills. Medical students’ perceptions and confidence in LLMs are associated with usage. While students see potential in reducing administrative burden, they also express concern about impacts on learning. Respondents highlighted topics that should be included in formal LLM education.