What question did this study set out to answer?

This study aims to evaluate the diagnostic accuracy and triage recommendations of AI models against neurologists for suspected strokes.

July 12, 2026Open Access

Bridging Artificial and Human Intelligence: Comparative Performance of AI Models and Neurologists in Stroke Recognition- A Cross-Sectional Study

Key Points

This study aims to evaluate the diagnostic accuracy and triage recommendations of AI models against neurologists for suspected strokes.
Cross-sectional study of 200 Reddit posts regarding stroke symptoms
Responses evaluated by two LLMs (ChatGPT-4, Gemini) and board-certified neurologists using 7-point Likert scales
Focus on criteria: Ease of Understanding, Scientific Adequacy, and Overall Satisfaction
Neurologists recommended ED visits more often (58.5%) compared to LLMs (45%) with p < 0.001.
AI models had significant 'Unable to determine' responses (ChatGPT: 11.5%, Gemini: 14.5%) not reported by neurologists (0%).
Neurologist responses were rated highest for Overall Satisfaction (Median 7) and Ease of Understanding (Median 7) with p < 0.001.

Abstract

Background Stroke is a major cause of disability and mortality worldwide. Thus, early detection and intervention, along with appropriate triage, are crucial. The development of large language models (LLMs), such as ChatGPT and Gemini, presents new potential for artificial intelligence in healthcare, including clinical decision support. The objective of this study was to evaluate the diagnostic accuracy and quality of triage recommendations from leading LLMs compared to those from board-certified neurologists in patients with suspected acute stroke. Methods This was a cross-sectional study of 200 posts in the Reddit “AskDocs” section related to possible symptoms of stroke. These posts elicited responses from two LLMs, ChatGPT-4 and Gemini, as well as two board-certified neurologists. Two experienced emergency medicine specialists, who were independent of the survey, evaluated responses for three criteria online using 7-point Likert scales: Ease of Understanding, Scientific Adequacy and Overall Satisfaction. The outcome of interest was advising an Emergency Department (ED) visit. Results Neurologists were much more willing to advocate for visiting the ED (58.5%) than ChatGPT (45%) or Gemini (45%) (p

AI에게 질문

Bookmark

View Full Paper