What question did this study set out to answer?

Evaluate ChatGPT's performance on radiology board-style examination questions and identify strengths and limitations.

May 16, 2023Radiology

Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations

View Full Paper

Ask AI

Bookmark

Evaluate ChatGPT's performance on radiology board-style examination questions and identify strengths and limitations.
Exploratory prospective study conducted from February 25 to March 3, 2023, with 150 multiple-choice questions.
Questions were categorized by lower-order and higher-order thinking and by topic (physics, clinical).
Performance assessed overall and by question type, with univariable analysis performed.
ChatGPT answered 69% of questions correctly (104 of 150), performing better on lower-order (84%) compared to higher-order questions (60%).
Worse performance noted on description of imaging findings (61%), calculation and classification (25%), and application of concepts (30%).
Consistently confident language used, even when incorrect (100%).

No takes yet. Share an insight, caveat, or question.

Exploratory study assesses ChatGPT's radiology exam performance, revealing strengths in lower-order questions but weaknesses in higher-order ones.

Bhayana et al. (2023) studied this question.

View Full Paper

Ask AI

Bookmark

Exploratory study assesses ChatGPT's radiology exam performance, revealing strengths in lower-order questions but weaknesses in higher-order ones.