A custom genAI-driven ECG learning app was highly rated by students (uMARS 4.57/5) but did not demonstrate a definitive short-term effect on exam performance compared to control sites.
Cohort
Yes
Does a genAI-developed custom ECG learning app improve exam performance in first-year medical students?
A genAI-developed ECG learning app was highly rated by medical students for usability but did not demonstrate a definitive short-term improvement in exam performance.
Generative artificial intelligence (genAI) enables educators to build custom learning tools, but the feasibility and impact of educator-driven, AI-assisted development (“vibe coding”) in medical education remain unclear. This study describes the rapid development of a custom ECG learning application using Gemini 3.1 Pro, evaluates its association with exam performance using difference-in-differences (DiD) and triple-difference (DDD) analyses, and assesses student perceptions with the user version of the Mobile App Rating Scale (uMARS). The app was implemented at one WWAMI site (intervention) with five sites as controls; aggregate performance from two first-year medical student cohorts (E24 vs. E25) was analyzed, comparing ECG-focused (focal) to non-ECG (baseline) exam items. DDD effects were inconsistent across exams, with no overall pooled effect on focal performance relative to baseline versus controls. In contrast, students rated the app highly (overall uMARS 4.57/5), particularly for quiz customization and waveform annotations. These findings support the feasibility of rapidly building and deploying tailored educational tools via genAI-assisted workflows and suggest strong perceived usability and acceptability among students. However, the study did not demonstrate a definitive short-term learning effectiveness effect on exam performance. Vibe coding is therefore positioned as a practical model for faculty-driven, context-specific educational innovation that requires further evaluation across broader implementations.
Janabi et al. (Tue,) conducted a cohort in Medical education (ECG interpretation). Custom genAI-driven ECG learning application vs. Five control sites without the application was evaluated on Exam performance on ECG-focused items relative to baseline and controls. A custom genAI-driven ECG learning app was highly rated by students (uMARS 4.57/5) but did not demonstrate a definitive short-term effect on exam performance compared to control sites.