Comparison of Radiologists and Multimodal Large Language Models Responses to Radiology ImageQuest

Motivation: What are the capabilities of multimodal large language models (LLMs) in addressing radiology-related questions, and can they enhance the performance of junior radiologists? Goal(s): To compare the performance of multimodal LLMs against radiologists of varying expertise levels and assess the impact of LLM assistance on junior radiologists' skills. Approach: This study evaluated the performance of multimodal LLMs against radiologists using the Radiology ImageQuest dataset, comprising 1,251 cases from six reputable sources. Results: Advanced LLMs like GPT-4o and Claude-3.5-sonnet demonstrated performance comparable to senior radiologists. Junior radiologists, with GPT-4o's assistance, nearly doubled their accuracy and achieved mid-level performance after a three-month period. Impact: Multimodal LLMs show promise in radiology education and practice, while further research is needed to validate their impact on real clinical applications