A multimodal framework integrating ambulatory ECG and clinical text embeddings consistently outperformed unimodal baselines for predicting sudden cardiac death and pump failure death.
Does a multimodal framework integrating ambulatory ECG and clinical features improve prediction of sudden cardiac death and pump failure death compared to unimodal baselines?
Integrating ambulatory ECG data with LLM-derived clinical text embeddings improves the prediction of sudden cardiac death and pump failure death compared to unimodal approaches.
Ambulatory electrocardiograms (ECG) provides continuous monitoring of the heart's electrical activity. However, many existing machine learning and artificial intelligence models for analyzing ambulatory ECG traces are often unimodal and do not incorporate patient clinical context. In this study, we propose a multimodal framework integrating ambulatory ECG-derived representations with clinical text embeddings to predict two cardiac outcomes: sudden cardiac death and pump failure death. Ambulatory ECG traces are preprocessed, segmented, and encoded via a multiple instance learning and temporal convolutional neural network framework. In parallel, patient clinical features are parsed into structured prompts, which are passed through a large language model to generate clinical reasoning; this reasoning passes through a biomedical language encoder to generate a text embedding. With the ECG and text embeddings, we systematically evaluate multiple fusion strategies, including concatenation- and gating-based approaches, to integrate these two data modalities. Our results demonstrate that multimodal models consistently outperform unimodal baselines, with adaptive fusion mechanisms providing the greatest improvements in predictive performance. Decision curve analysis highlights the potential clinical utility of the proposed framework for risk stratification. Finally, we visualize model attention across modalities, including ECG attention patterns, segment-level saliency, heart rate variability features, and clinical reasoning, to contextualize patient-specific predictions.
Swee et al. (Wed,) conducted a other in Sudden cardiac death and pump failure death. Multimodal framework integrating ambulatory ECG and clinical text embeddings vs. Unimodal baselines was evaluated on Sudden cardiac death and pump failure death prediction. A multimodal framework integrating ambulatory ECG and clinical text embeddings consistently outperformed unimodal baselines for predicting sudden cardiac death and pump failure death.