Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions

Yasin Celal Güneş; Mehmet Ülkir

doi:10.32708/uutfd.1568479

January 12, 2025Uludağ Üniversitesi Tıp Fakültesi DergisiOpen Access

Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions

View Full Paper

Ask AI

Bookmark

Share

Key Points

To compare the diagnostic accuracy of four multimodal large language models against a radiologist and an anatomist in answering multiple-choice visual neuroanatomy questions.
Cross-sectional evaluation assessing performance across 100 visual multiple-choice neuroanatomy questions sourced from Radiopaedia.
Tested four multimodal LLMs (GPT-4V, GPT-4o, LLaVA, and Gemini 1.5 Flash), an anatomist, and a radiologist, analyzing comparative accuracy using the McNemar test.
The radiologist demonstrated the highest accuracy at 90%, significantly outperforming the anatomist who achieved 67%.
Among multimodal LLMs, GPT-4o performed best with 45% accuracy, followed by Gemini 1.5 Flash (35%), GPT-4V (22%), and LLaVA (15%).

No abstract yet

We only have this paper's title and publication details.

Read it at the source

Implication

Cross-sectional study demonstrates superior visual neuroanatomy diagnostic accuracy by radiologists over multimodal AI models, indicating substantial performance gaps in clinical imaging...

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Güneş et al. (2025) studied this question.

synapsesocial.com/papers/6a12886b4a30fb84f7a58b38 https://doi.org/10.32708/uutfd.1568479

View Full Paper

Ask AI

Bookmark

Share

Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions

Key Points

No abstract yet

Implication

Discussion

Cite This Study

Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions

Key Points

No abstract yet

Implication

Discussion

Cite This Study

Keep reading

More related papers

More from Uludağ Üniversitesi Tıp Fakültesi Dergisi

Keep reading

More related papers

More from Uludağ Üniversitesi Tıp Fakültesi Dergisi