Effect of a Clinical Decision Support System on Team‐Based Clinical Quiz Performance Among Japanese Medical Students: A Pre–Post Pilot Study

Background Educational impact of point‐of‐care clinical decision support system (CDSS) on medical students remains insufficiently studied. Our aim was to evaluate the effect of a CDSS on team‐based clinical quiz performance and confidence in using medical information resources. Methods We conducted a nationwide, online, pre–post quasi‐experimental pilot study on November 24, 2024. Eighteen teams of Japanese medical students (team size ranging from one to three members) completed ten multiple‐choice questions based on two clinical cases. Teams first answered questions using any resources except UpToDate (Pre phase), then re‐answered the same questions using UpToDate only (Post phase) under stricter time constraints. The primary outcome was team‐based quiz score. Secondary outcomes were team‐based confidence in using UpToDate and other resources. Paired t ‐tests and Wilcoxon signed‐rank tests were performed. Sensitivity analyses addressed missing post‐intervention confidence data. Results Seventeen teams (43 individuals) were included in the analysis. Mean quiz scores increased from 4.00 (SD 1.70) to 4.76 (SD 2.17) (paired t ‐test p = 0.01; Wilcoxon p = 0.026). Confidence in using UpToDate improved from 1.38 (SD 0.77) to 2.54 (SD 0.97) (paired t‐test p = 0.0021), while confidence in other resources showed no significant change. Sensitivity analyses supported the confidence improvement under neutral and best‐case assumptions but not under the extreme worst‐case scenario. Conclusions CDSS use may be associated with modest improvements in team‐based quiz performance and confidence among medical students in this volunteer‐based sample; generalizability to broader populations requires further investigation. Future studies should examine learning outcomes, clinical applicability, and curricular integration.