Why the study?
Diagnosing myocarditis requires integrating multimodal CMR and clinical data requiring expertise, prompting investigation into GPT-4's performance for report-based medical decision-making in suspected myocarditis.
Does GPT-4 provide comparable diagnostic accuracy to human readers for detecting myocarditis based on CMR reports and clinical data in patients with suspected myocarditis?
Does GPT-4 provide comparable diagnostic accuracy to human readers for detecting myocarditis based on CMR reports and clinical data in patients with suspected myocarditis?
GPT-4 demonstrates diagnostic accuracy comparable to a first-year radiology resident for detecting myocarditis from CMR reports and clinical data, highlighting its potential as a diagnostic decision-support tool.
Answers are grounded in this paper and cite their sources.
May support AI-assisted CMR reporting for myocarditis; leaves open prospective validation before clinical use.
BACKGROUND: Diagnosing myocarditis relies on multimodal data, including cardiovascular magnetic resonance (CMR), clinical symptoms, and blood values. The correct interpretation and integration of CMR findings require radiological expertise and knowledge. We aimed to investigate the performance of Generative Pre-trained Transformer 4 (GPT-4), a large language model, for report-based medical decision-making in the context of cardiac MRI for suspected myocarditis. METHODS: This retrospective study includes CMR reports from 396 patients with suspected myocarditis and eight centers, respectively. CMR reports and patient data including blood values, age, and further clinical information were provided to GPT-4 and radiologists with 1 (resident 1), 2 (resident 2), and 4 years (resident 3) of experience in CMR and knowledge of the 2018 Lake Louise Criteria. The final impression of the report regarding the radiological assessment of whether myocarditis is present or not was not provided. The performance of Generative pre-trained transformer 4 (GPT-4) and the human readers were compared to a consensus reading (two board-certified radiologists with 8 and 10 years of experience in CMR). Sensitivity, specificity, and accuracy were calculated. RESULTS: GPT-4 yielded an accuracy of 83%, sensitivity of 90%, and specificity of 78%, which was comparable to the physician with 1 year of experience (R1: 86%, 90%, 84%, p = 0.14) and lower than that of more experienced physicians (R2: 89%, 86%, 91%, p = 0.007 and R3: 91%, 85%, 96%, p < 0.001). GPT-4 and human readers showed a higher diagnostic performance when results from T1- and T2-mapping sequences were part of the reports, for residents 1 and 3 with statistical significance (p = 0.004 and p = 0.02, respectively). CONCLUSION: GPT-4 yielded good accuracy for diagnosing myocarditis based on CMR reports in a large dataset from multiple centers and therefore holds the potential to serve as a diagnostic decision-supporting tool in this capacity, particularly for less experienced physicians. Further studies are required to explore the full potential and elucidate educational aspects of the integration of large language models in medical decision-making.
No takes yet. Share an insight, caveat, or question.
Kaya et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: