A Comparative Analysis of The Diagnostic Efficacy of Diverse Artificial Intelligence (AI) Algorithms in Ultrasound-Based Cases

Aim To evaluate the diagnostic performance of Large Language Models (LLM) (ChatGPT 3.5, ChatGPT 4, Gemini 1.0, and Gemini Advance) in Ultrasound (US) cases and their superiority over each other Materials and Methods In this retrospective study, the data of 20 real cases with US examination and confirmed diagnoses were evaluated between 2020-2024. Clinical information, relevant laboratory data, and US findings of these cases were simultaneously presented to four Artificial Intelligence (AI) (ChatGPT 3.5, ChatGPT 4, Gemini 1.0, Gemini Advance). The correct response rates of the four AIs to the cases were compared. Two radiology experts in the US evaluated the answers. Results The correct response rates of ChatGPT 3.5, ChatGPT 4, Gemini 1.0, and Gemini Advance models in the cases were 92% (23/25), 92% (23/25), 76% (19/25), 84% (21/25), respectively, and with no statistically significant differences between them. Conclucion This is the first study about four AI performances in diagnosis in real US cases. The results suggest that no matter which AI we use, AIs have the potential to assist radiologists in diagnosis significantly. The fact that they are easy and fast to use can also significantly speed up the daily workflow. However, it should be remembered that they cannot yet completely replace a radiologist.