Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
July 24, 2024The Lancet Digital HealthOpen Access

The diagnostic and triage accuracy of the GPT-3 artificial intelligence model: an observational study

View Full Paper
Ask AI
Bookmark
Share

Authors

DLDavid M. LevineRTRudraksh TuwaniBKBenjamin Kompa

Discussion

Loading...

Member takes

Overview

Observational study demonstrates that GPT-3 achieves near-physician diagnostic accuracy but inferior triage across clinical vignettes, highlighting current safety limits.

Key Points

  • The diagnostic accuracy of GPT-3 reached 88% in top-three differentials, significantly outperforming lay individuals while trailing practicing physicians at 96%.
  • Observational study using bootstrap resampling assessed model calibration across 48 synthetic case vignettes, revealing an inverse relationship with case acuity.
  • Highlights safety risks in autonomous triage advice, as GPT-3 deprioritised truly emergent cases and produced Brier scores reflecting poorer calibration than physicians.

Cite This Study

Levine et al. (2024) studied this question.

synapsesocial.com/papers/68e5f2d2b6db64358758754ehttps://doi.org/10.1016/s2589-7500(24)00097-9
View Full Paper
Ask AI
Bookmark
Share

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1Performance of emergency triage prediction of an open access natural language processing based chatbot application (ChatGPT)2023 · 63 citations
  2. 2Accuracy of a Commercial Large Language Model (ChatGPT) to Predict the Diagnosis for Pre-Hospital Patients Suitable for Ambulance Dispatch: Diagnostic Accuracy Study2026
  3. 3ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis2024 · 90 citations
  4. 4Can ChatGPT Take My Call? evaluating AI in gynecologic oncology telephone triage2026
  5. 5Assessing the utility of artificial intelligence throughout the triage outpatients: a prospective randomized controlled clinical study2024 · 4 citations