Key result
MedGemma 1.5 misses all acute coronary occlusion ECGs, assigning high concern in 0% vs. 100%.
Why the study?
Medical vision-language models performance for ECG identification of acute coronary occlusion remains poorly characterised outside benchmark datasets.
Does a medical-domain vision-language model accurately detect acute coronary occlusion on 12-lead ECGs compared to a general-purpose model?
Does a medical-domain vision-language model accurately detect acute coronary occlusion on 12-lead ECGs compared to a general-purpose model?
Absolute Event Rate: 0% vs 100%
A medical vision-language model demonstrated clinically important failures in detecting acute coronary occlusion on ECGs, highlighting the critical need for task-specific validation before deploying multimodal AI in time-critical cardiovascular settings.
VLM testing on limited ACO ECGs is hypothesis-generating; leaves open clinical utility pending larger prospective validation.
Introduction: Vision-language models (VLMs) are increasingly explored for clinical interpretation tasks. Electrocardiographic (ECG) identification of acute coronary occlusion (ACO) is a high-risk application requiring accurate pattern recognition, and medical VLM performance on this task remains poorly characterised outside benchmark datasets.,Methods: A medical-domain VLM (MedGemma 1.5) was tested between 30 January and 22 February 2026 using 15 de-identified 12‑lead ECGs: 14 source-labelled ACO or high-risk occlusion phenotypes and one normal control. Each ECG was submitted with a standardised prompt requesting a structured interpretation and overall concern level (low, intermediate, or high), without iterative prompting. The same ECGs were submitted to a general-purpose VLM comparator (Gemini 3.1 Pro).,Results: The model assigned low concern in 8/15 cases (53%) and intermediate concern in 7 (47%); no case reached high concern, including among the 14 high-risk ECGs. It under-recognised ST-segment elevation, failed to integrate reciprocal and territorial patterns, and missed hyperacute T-wave morphology. Conduction abnormalities and quantitative parameters were assessed inconsistently. The comparator assigned high concern to all 14 high-risk cases but also to the normal control, citing an incorrect ECG pattern.,Conclusion: A medical VLM showed clinically important interpretation failures and did not escalate concern in any high-risk ECG. A general-purpose comparator was more consistent across high-risk cases but misclassified the normal control, suggesting some failure modes may not be modelspecific. These findings support task-specific validation and governance before multimodal AI is deployed in time-critical cardiovascular settings.
No takes yet. Share an insight, caveat, or question.
Bishop et al. (2026) studied acute coronary occlusion (n=15). MedGemma 1.5 vs. Gemini 3.1 Pro was evaluated on overall concern level (low, intermediate, or high). The medical VLM MedGemma 1.5 assigned 0% high concern to 14 high-risk acute coronary occlusion ECGs, while the general-purpose comparator assigned high concern to all cases including the control.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: