Mobile Learning in Medical Education: A Realist Evaluation of Usage, Context, and Exam Performance in a Curricular Setting (Preprint)

BACKGROUND Mobile learning (mLearning) is widely used in higher education but research remains fragmented and context-insensitive. In undergraduate medical education, in which knowledge acquisition must align with competency-based training, mLearning may bridge formal and informal learning. Previous research has focused on two main areas: technology acceptance or intervention effectiveness, but rarely their integration. Using realist evaluation (RE), this study investigated the conditions under which mLearning supports outcomes in authentic curricula settings. OBJECTIVE This study aimed to (1) assess the impact of a voluntary mLearning intervention on exam performance and (2) identify learner profiles and context–mechanism–outcome (CMO) patterns explaining differential engagement and outcomes. METHODS A quasi-experimental study was conducted among fifth-semester undergraduate medical students at a German medical school across two consecutive summer semesters (2023 and 2024). A voluntary, app-based mLearning course in “Microbiology, Hygiene, and Virology” was delivered via the eSquirrel platform, featuring interactive tasks, gamification, and spaced-repetition features. Data sources included non-reactive app usage logs, baseline academic performance, demographics, and post-semester survey responses. Cluster analysis identified usage profiles, and CMO patterns were derived by triangulating usage, performance, and survey data. Neither randomization nor blinding was applied, to capture real-world curricular use. RESULTS Of 245 eligible students, 220 (89.8%) participated; 110 (50%) used the app. In 2024 app users (n=64) outperformed non-users (n=46) in the oral microbiology exam (mean grade 2.3 vs 2.8; t₆₃.₀=1.90, one-sided P=.031). After adjustment, these differences were largely explained by baseline academic performance, with limited evidence for an independent app effect. Cluster analysis of app users identified three engagement profiles: (1) continuous low-intensity users (n=60), (2) intensive, exam-oriented users (n=31), and (3) early, disengaging users (n=19). Cluster 2 reported the highest satisfaction, perceived learning gains, and exam performance. A refined program theory suggested that when learners with sufficient baseline ability and strategic study habits engage “just-in-time” (Context), mechanisms such as motivation, spaced repetitions, microlearning, and gamified feedback (Mechanism) lead to improved exam performance (Outcome). CONCLUSIONS The effectiveness of mLearning is not universal but varies with learner context and engagement strategy. Strategic, exam-oriented use—driven by motivation, enjoyment, and self-regulation—yielded the strongest benefits. Passive usage data provided valuable insights into authentic learning behavior, while quasi-experimental designs require contextual adjustment to clarify causal relationships. Effective mLearning interventions should be both educationally useful and engaging, fostering enjoyment alongside learning outcomes. Future research should address equity concerns, as higher-performing students benefited most, and explore adaptive approaches to support diverse learners.