July 6, 2023Postgraduate Medical Journal

Artificial intelligence in orthopaedics: can Chat Generative Pre-trained Transformer (ChatGPT) pass Section 1 of the Fellowship of the Royal College of Surgeons (Trauma & Orthopaedics) examination?

View Full Paper

Ask AI

Bookmark

Why the study?

AI in healthcare remains in its infancy, requiring scrutiny of its reliability, specifically whether ChatGPT could pass Section 1 of the FRCS examination in Trauma and Orthopaedic Surgery.

Can ChatGPT pass Section 1 of the Fellowship of the Royal College of Surgeons (Trauma & Orthopaedics) examination?

Population

Papers 1 and 2 of UKITE 2022 directly inputted into ChatGPT

Comparison

ChatGPT performance vs FRCS pass rate and human candidates

Design

Examination evaluation study

Authors

RCRory CuthbertRoyal National Orthopaedic Hospital ASAshley I. SimpsonRoyal National Orthopaedic Hospital

Discussion

Member takes

Implication

Warrants caution using current AI for FRCS (T&O) preparation; leaves open whether future models can reach specialist thresholds.

Structured PICO

Can ChatGPT pass Section 1 of the Fellowship of the Royal College of Surgeons (Trauma & Orthopaedics) examination?

Population

Chat Generative Pre-trained Transformer (ChatGPT) artificial intelligence model

Intervention

Answering Papers 1 and 2 of the UK and Ireland In-Training Examination (UKITE) 2022 (surrogate for the Fellowship of the Royal College of Surgeons [FRCS] examination in Trauma and Orthopaedic Surgery)

Comparator

FRCS pass rate and mean score achieved by human candidates

Outcome

Examination score

ChatGPT is currently unable to pass the FRCS (Trauma & Orthopaedics) examination and fails to recognize its own limitations, highlighting the need for caution when using it as a medical resource.

Cite This Study

Cuthbert et al. (2023) studied this question.

synapsesocial.com/papers/69df01bab1b9c2de2006b0b2 https://doi.org/10.1093/postmj/qgad053

View Full Paper

Ask AI

Bookmark