Answers are grounded in this paper and cite their sources.
Cross-sectional evaluation reveals conversational AI improves usability in medical education, suggesting further development is needed.
BACKGROUND Conversational artificial intelligence (AI) systems offer potential solutions to traditional constraints in medical consultation skills training, including high costs, scheduling difficulties, and varied standardisation. OBJECTIVE There is limited evidence evaluating medical professionals' perceptions of AI generated patient interactions across multiple fidelity dimensions and assess the educational value of conversational AI for consultation skills training. METHODS Cross-sectional evaluation study at a UK medical school involving 179 participants (medical students and general practitioners) at 70 sites. Participants completed standardised clinical scenarios using SimFlow.ai conversational AI system, followed by structured questionnaires evaluating AI Realism, Medical Content, Educational Value, Feedback, and Usability domains. Data were analysed using descriptive statistics and non-parametric tests to assess domain performance and participant characteristics. RESULTS Medical Content received highest ratings (median 4.0, IQR 1.0 [4.0-5.0]), with 97.9% rating clinical plausibility highly. Educational Value was rated positively (IQR 1.0-2.0 [3.0-4.0] though AI Realism received moderate scores (median 3.0, IQR 2.0 [2.0-4.0]). Participants with prior AI experience gave significantly higher ratings for AI Realism than those without prior experience (mean 3.81 vs 3.07, p<0.05). Qualitative analysis revealed four themes: clinical authenticity, interactional limitations, educational potential, and implementation considerations. CONCLUSIONS Conversational AI demonstrates strong capabilities in functional fidelity (clinical accuracy) despite limitations in conversational fidelity (realism). The technology shows promise for supplementary clinical skills training rather than higher-stakes assessment, with future development needed in dialogue naturalness and feedback capabilities. CLINICALTRIAL https://doi.org/10.17605/OSF.
No takes yet. Share an insight, caveat, or question.
Jacobs et al. (2025) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: