Perceived Ratings of AI-Generated and Expert-Reviewed vs Clinician-Developed Virtual Cardiology Patient Scenarios Among Medical Students: Randomized Controlled Study

BACKGROUND Virtual patients (VPs) are widely used in medical education, but creating high-quality cases requires clinical expertise, iterative revision, and alignment with educational objectives. Generative AI may support VP authoring, but direct comparisons between AI-generated and clinician-developed cases delivered within the same educational setting remain limited. OBJECTIVE This study aimed to compare medical students’ learner-perceived ratings of 1 AI-generated, expert-reviewed virtual cardiology patient scenario with those of 1 clinician-developed virtual cardiology patient scenario delivered through the same mobile platform. METHODS A randomized, participant-blinded 2-arm controlled study was conducted in October 2025 during scheduled fifth-year emergency medicine course sessions. Eligible participants were fifth-year medical students who had completed the cardiology course. Students were assigned 1:1 to complete either a clinician-developed VP (CD-VP; n=64) or a ChatGPT-generated VP (GPT-VP; n=67). Both scenarios followed a shared acute coronary syndrome blueprint and were delivered through the same mobile-optimized app (mobile VPs app [MobiViP]) with identical interface structure and branching logic. The GPT-VP was generated using GPT-5 and refined through iterative follow-up prompting under expert review; direct manual rewriting was prohibited. After completion, students completed a mandatory 19-item questionnaire covering realism and validity, cognitive load, self-reported learning and reflection, and engagement and satisfaction. Analyses included reliability testing, 1-sample tests against a neutral midpoint, Mann-Whitney U tests, and Spearman correlations. Full scenario content and development details are provided as multimedia appendices. RESULTS A total of 131 complete questionnaires were analyzed, with no item-level missing data. Mean subscale scores in the CD-VP group were 3.81 (SD 0.64) for realism and validity, 3.67 (SD 0.74) for cognitive load, 3.64 (SD 0.73) for self-reported learning and reflection, and 4.16 (SD 0.64) for engagement and satisfaction; corresponding GPT-VP scores were 4.11 (SD 0.56), 3.03 (SD 0.76), 3.86 (SD 0.77), and 4.28 (SD 0.58), respectively. Compared with the CD-VP, the GPT-VP received higher ratings for realism and validity (Mann-Whitney U=1503.0; P=.003) and lower cognitive load ratings, indicating a lower perceived burden (Mann-Whitney U=1070.5; P<.001). Self-reported learning and reflection showed a marginal difference between groups, with a higher rating for the GPT-VP (Mann-Whitney U=1719.0; P=.05), whereas engagement and satisfaction did not differ significantly (Mann-Whitney U=1922.5; P=.31). Internal consistency was acceptable to good for most subscales, although internal consistency for the realism and validity subscale in the GPT-VP group was moderate (Cronbach α=0.67). CONCLUSIONS In this single-center randomized study, 1 AI-generated, expert-reviewed cardiology VP scenario was rated similarly to or more favorably than 1 clinician-developed scenario in selected learner-perceived domains. Future research should expand to objective learning, retention, development workload, and scalability outcomes.