What does this research mean for the field?

ChatGPT models, particularly version 4.0, demonstrate excellent performance on medical school examinations, outperforming fourth-year medical students and showing improved accuracy upon re-prompting. Novelty: ClaimNovelty.INCREMENTAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

To evaluate the problem-solving accuracy, reliability, and answer regeneration performance of ChatGPT models (3.5 and 4.0) on medical school examination questions compared with medical students.

August 23, 2026MedicineOpen Access

Artificial intelligence in medical education

View Full Paper

Ask AI

Bookmark

Authors

KKKyung Wook Kim HJHong Bae Jeon JCJong Hyuk Choi

Discussion

Member takes

Overview

Comparative evaluation demonstrates superior examination performance by GPT-4 relative to medical students, indicating strong diagnostic capabilities alongside educational utility.

Key Points

To evaluate the problem-solving accuracy, reliability, and answer regeneration performance of ChatGPT models (3.5 and 4.0) on medical school examination questions compared with medical students.
Administered 49 text-based multiple-choice questions spanning preventive medicine and psychiatry to 41 fourth-year medical students and both ChatGPT models (GPT-3.5 and GPT-4.0).
Assessed primary accuracy on the initial prompt and evaluated answer regeneration performance by re-prompting the models following incorrect responses.
Fourth-year medical students achieved a mean score of 29.5 ± 5.9 out of 49.
Initial scores were 28 for GPT-3.5 and 43 for GPT-4.0, which increased upon re-prompting to 40 and 46, respectively, with GPT-4.0 outperforming all individual students after re-prompting.
Both AI models demonstrated significantly higher accuracy than students on easy questions, whereas performance differences between students and the initial GPT-3.5 prompt on difficult questions were nonsignificant.

Cite This Study

Kim et al. (2026) studied this question.

synapsesocial.com/papers/6a8aadb47677a3411444615f https://doi.org/10.1097/md.0000000000050323

View Full Paper

Ask AI

Bookmark

August 23, 2026MedicineOpen Access

Artificial intelligence in medical education

View Full Paper

Ask AI

Bookmark

Authors

KKKyung Wook Kim HJHong Bae Jeon JCJong Hyuk Choi

Discussion

Member takes

Overview

Comparative evaluation demonstrates superior examination performance by GPT-4 relative to medical students, indicating strong diagnostic capabilities alongside educational utility.

Key Points

To evaluate the problem-solving accuracy, reliability, and answer regeneration performance of ChatGPT models (3.5 and 4.0) on medical school examination questions compared with medical students.
Administered 49 text-based multiple-choice questions spanning preventive medicine and psychiatry to 41 fourth-year medical students and both ChatGPT models (GPT-3.5 and GPT-4.0).
Assessed primary accuracy on the initial prompt and evaluated answer regeneration performance by re-prompting the models following incorrect responses.
Fourth-year medical students achieved a mean score of 29.5 ± 5.9 out of 49.
Initial scores were 28 for GPT-3.5 and 43 for GPT-4.0, which increased upon re-prompting to 40 and 46, respectively, with GPT-4.0 outperforming all individual students after re-prompting.
Both AI models demonstrated significantly higher accuracy than students on easy questions, whereas performance differences between students and the initial GPT-3.5 prompt on difficult questions were nonsignificant.

Cite This Study

Kim et al. (2026) studied this question.

synapsesocial.com/papers/6a8aadb47677a3411444615f https://doi.org/10.1097/md.0000000000050323

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

View Full Paper

Ask AI

Bookmark