EXPRESS: Medical Students' Perceptions of AI-Generated Practice Questions as Learning Tools

Generative artificial intelligence (GenAI) tools, including large language models (LLMs) such as ChatGPT, have potential as educational adjuncts to enhance student learning. This study evaluated the perceived utility of and performance outcomes associated with formative, AI-generated, USMLE-style practice questions among preclinical medical students. Multiple-choice questions (MCQs) aligned with 15 microbiology and endocrinology lectures were generated with ChatGPT 4.0 and distributed via Google Forms to 386 students (198 MS1, 188 MS2) at a U.S. medical school. Each question set consisted of 6 questions on average, and these groupings were considered individual “question sets” in our analysis. Question completion was optional for students and a total of 490 question sets were completed. Students provided feedback on 94.9% of sets, with 82.8% rating the questions as “Helpful,” 16.1% as “Somewhat Helpful,” and 1.1% as “Not Helpful.” MS2 students answered a significantly higher number of questions correctly relative to MS1 students (84.4% vs. 78.6%, p < .001), and performance varied by lecture topic. However, average scores did not differ significantly based on whether students provided feedback or how they rated the questions (p = .587). These findings suggest that AI-generated MCQ questions are well-received by students as a formative learning tool and may serve as scalable, curriculum-aligned tools to support self-directed learning in medical education.