Heart failure is a prevalent condition associated with high morbidity and mortality. The rapid evolution and widespread adoption of artificial intelligence applications have significantly advanced in the field of medicine. We aim to assess the value and reliability of ChatGPT 4o, Gemini, and Bing Chat’s responses regarding heart failure. A list of fifty commonly asked questions regarding heart failure was asked twice to ChatGPT 4o, Gemini, and Bing Chat. Two experienced cardiologists assessed each the natural language processing models’ answers without knowing each other’s scores. The content of answers was evaluated using the following scale: correct (1), partially correct (2), a mix of accurate and inaccurate (3) and completely inaccurate (4). Most answers were correct or partially correct; only Bing Chat gave some inaccurate responses, unlike ChatGPT-4o and Gemini. In terms of ‘correct’ answers, ChatGPT scored 88%, Gemini scored 70%, and Bing Chat scored 64%. ChatGPT 4o provided the highest ‘reproducible’ score at 88%, followed by Gemini at 86% and Bing Chat at 72%. This study demonstrated that ChatGPT 4o, in particular, has the ability to produce valuable responses to patient inquiries regarding heart failure. In the future, as chatbots are further investigated and improved, these models may have the potential to be utilized by both patients and healthcare professionals for managing chronic conditions like heart failure.
Çetin et al. (Wed,) studied this question.