Feasibility of an AI-simulated patient encounter delivered in group and individual learning formats for first-year medical students: a two-cohort pilot study

Generative artificial intelligence (AI) and large language model (LLM)–based simulated patients are being evaluated for problem-based learning and case-based learning in undergraduate medical education, but delivery formats are often under-specified and outcome measurement is inconsistently reported. We assessed the feasibility and acceptability of a single LLM-driven, AI-simulated patient encounter delivered in two formats—a facilitated whole-group session and a self-paced individual session—to first-year medical students (MS1), and we describe the development and provisional status of a learner-perception survey instrument. Two non-overlapping, non-randomized MS1 cohorts at one United States medical school completed the same case (Group, n = 7; Individual, n = 12) and the Behavioral Safety Language Scale–Provisional (BSLS-P) before and after the encounter. Pre-to-post change was tested with Wilcoxon signed-rank exact tests, with Benjamini–Hochberg false discovery rate (FDR) correction across subscale comparisons. Feasibility outcomes (recruitment, encounter completion, survey completion, technical failures, and encounter duration) and acceptability outcomes (post-encounter engagement ratings, delivery-format preference, and open-ended comments) are reported descriptively. Of 72 students invited, 19 (26%) enrolled; all completed the encounter and paired pre/post surveys (100%), with no technical failures (no encounter was terminated and no participant was excluded), although learners reported intermittent technical problems that interrupted but did not end encounters; the group session lasted approximately 35 min and individual encounters a median of 31 min (range 17–83). Pre-to-post shifts were small and imprecise; in the Group cohort, median engagement changed by + 0.33 points (Wilcoxon r = 0.13, 95% CI 0.00 to 0.90). The largest effect—a psychological-safety gain in the Individual cohort (Wilcoxon r = 0.62)—was significant before correction ( P =0.026) but not after (adjusted P =0.20). No between-cohort comparison reached significance. A single motivation item declined by one median point in both cohorts. The modal post-encounter preference was a hybrid “solo first, then group” model (7/12, 58%). On the feasibility and acceptability indicators examined, the encounter was delivered as planned in both formats and was rated favorably by learners; these data do not address educational effectiveness. Observed pre-to-post shifts are exploratory and require confirmation in an adequately powered, randomized sample. The provisional instrument is described to support that work.