Key result
Multimodal LLMs fail to reliably identify ECGs warranting emergent catheterization due to poor specificity.
Why the study?
Do multimodal large language models accurately predict emergent catheterization laboratory activation from prehospital ECGs in patients with suspected acute myocardial infarction?
Observational (n=270)
Do multimodal large language models accurately predict emergent catheterization laboratory activation from prehospital ECGs in patients with suspected acute myocardial infarction?
General-purpose multimodal large language models are currently unreliable for predicting emergent catheterization laboratory activation from prehospital ECGs due to poor specificity and excessive false-positive rates.
Current multimodal LLMs risk excessive false activations if used for STEMI triage; leaves open whether domain-specific models can exceed machine algorithm specificity.
Background Rapid and accurate electrocardiogram (ECG) interpretation is essential for timely identification of ST-elevation myocardial infarction (STEMI) and activation of reperfusion pathways in emergency care. Objectives To evaluate the diagnostic performance of multimodal LLMs in identifying prehospital ECGs warranting emergent catheterization laboratory activation. Methods We performed a retrospective analysis of 615 ECGs from 270 emergency medical service patient encounters (EMS) with concern for acute myocardial infarction. The reference standard was cardiology activation of the STEMI pathway for emergent angiography. LLM-based image interpretation (three models) and ECG machine algorithm interpretations were compared. Sensitivity, specificity, positive predictive value, negative predictive value, and overall accuracy were calculated. Results Gemini demonstrated the highest sensitivity (95.3%; 95% CI 91.7–97.3) but extremely poor specificity (9.4%), indicating a high false-positive rate. ChatGPT and Claude showed moderate sensitivity (68.1% and 67.2%) with limited specificity (42.3% and 46.5%). The ECG machine algorithm demonstrated more balanced performance, with sensitivity of 67.7% (95% CI 61.4–73.4) and higher specificity (64.2%) than all LLMs. Conclusions Multimodal LLM interpretation of prehospital ECGs demonstrated clinically unreliable performance for identifying ECGs warranting emergent cardiac catheterization laboratory activation when benchmarked against real-world cardiology activation decisions. Although some models achieved high sensitivity, poor specificity resulted in excessive false-positive activation recommendations. These findings suggest that general-purpose LLMs are not appropriate for ECG-based catheterization laboratory activation decisions in time-sensitive cardiopulmonary care workflows.
No takes yet. Share an insight, caveat, or question.
Legendre et al. (2026) conducted an observational in Acute myocardial infarction (n=270). Multimodal large language models (Gemini, ChatGPT, Claude) vs. ECG machine algorithm interpretations was evaluated on Sensitivity for identifying prehospital ECGs warranting emergent catheterization laboratory activation (95% CI 91.7-97.3). Multimodal LLMs demonstrated unreliable performance for identifying ECGs warranting emergent catheterization, with Gemini achieving high sensitivity (95.3%) but poor specificity (9.4%).
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: