AI-powered simulated patients with automated feedback for enhancing headache history-taking skills: a convergent mixed-methods study

Medical students often struggle with headache disorders, which are common but diagnostically challenging due to limited clinical exposure and feedback during early clinical training. This study evaluated the feasibility and effectiveness of an AI-powered simulated patient (AISP) chatbot in enhancing history-taking and clinical reasoning for three primary headache disorders—migraine, tension-type, and cluster headache—among medical students at a tertiary university hospital in Thailand. We developed a simulated patient platform that provided automated, rubric-based feedback across three headache cases. Third-year medical students who had completed pre-clerkship requirements participated in a mixed-methods experimental study with a pre-post design. The study consisted of a 1-week self-study phase followed by a pre-test Objective Structured Clinical Examination (OSCE), a 1-week washout period, and a 1-week AISP practice phase followed by a post-test OSCE. Quantitative outcomes included changes in OSCE scores, Clinical Reasoning Indicator-History Taking (CRI-HT) scores, usability/satisfaction measured using the Chatbot Usability Questionnaire (CUQ). Focus group discussions (FGDs) explored participants’ learning experiences, perceived system limitations, and recommendations for future implementation. Qualitative data were analyzed thematically. Twenty-five medical students participated. OSCE scores increased by 22.5 points (p < 0.001, Cohen’s d = 1.93), and CRI-HT scores increased by 8.8 points (p < 0.001, Cohen’s d = 1.81). CUQ findings indicated high usability. Students commonly cited the platform’s intuitive interface and informative feedback, although some noted the chatbot’s responses were overly robotic. FGDs further highlighted three themes: (1) convenient, structured practice support learning; (2) dialogue felt realistic but was limited for practicing communication skills; and (3) students recommended greater case variety and integration into the curriculum. An AISP platform with automated, rubric-based feedback was feasible and associated with improvements in headache history-taking and clinical reasoning. It serves as a valuable complement to traditional teaching by enabling structured, repeated practice with timely feedback.