Cardio-respiratory deep learning model to predict hospital admission from emergency department electrocardiogram images and respiration videos: a prospective study

Background Early prediction of hospital admission at the emergency department (ED) triage can improve patient flow and resource allocation. Most existing models rely solely on structured data. Incorporating multimodal physiologic information, such as cardiac and respiratory signals, may better capture subtle clinical signs. This study aimed to develop a deep learning fusion model that integrates electrocardiogram (ECG) images and respiration videos to predict hospital admission at ED triage. Methods We prospectively collected ECG images and respiration videos from adult ED patients in a tertiary medical center. The proposed model consisted of modality-specific branches: an ECG branch using transfer learning, a respiration-signal-estimation pipeline, and a spatial attention module to enhance cardiopulmonary feature extraction. Class-balanced loss was applied to address dataset imbalance. Model performance was compared with the baseline Taiwan Triage Acuity Score (TTAS) and single-modality variants. Attention heatmaps using score-class activation mapping (score-CAM) were analyzed for interpretability. Results The study included 118 patients, with a 27% admission rate. The ECG-respiration fusion model achieved an accuracy of 0.814, a recall of 0.749, a precision of 0.635, an F1-score of 0.680, an area under the receiver operating characteristic curve (AUROC) of 0.863, and an area under the precision-recall curve (AUPRC) of 0.675, outperforming both TTAS and single-modality models. Ablation analyses confirmed the additive value of spatial attention and class-balanced loss. Score-CAM visualizations indicated the model focused on physiologically relevant features, such as respiratory waveform inflection points. Conclusions A multimodal deep learning approach combining ECG and respiration video signals can predict hospital admission at ED triage. Integrating such models into triage systems may enhance earlier risk identification and resource prioritization. Clinical trial number Not applicable.