Multimodal Signal Processing of ECG Signals with Time-Frequency Representations for Arrhythmia Classification

Arrhythmias are a frequently occurring phenomenon in clinical practice, but how to accurately distinguish subtle rhythm abnormalities remains an ongoing difficulty faced by the entire research community when conducting ECG-based studies. From a review of existing studies, two main factors appear to contribute to this problem: the uneven distribution of arrhythmia classes and the limited expressiveness of features learned by current models. To overcome these limitations, this study proposes a dual-path multimodal framework, termed DM-EHC (Dual-Path Multimodal ECG Heartbeat Classifier), for ECG-based heartbeat classification. The proposed framework links 1D ECG temporal features with 2D time–frequency features. By setting up the dual paths described above, the model can process more dimensions of feature information. The MIT-BIH arrhythmia database was selected as the baseline dataset for the experiments. Experimental results show that the proposed method outperforms single modalities and performs better for certain specific types of arrhythmias. The model achieved mean precision, recall, and F1 score of 95.14%, 92.26%, and 93.65%, respectively. These results indicate that the framework is robust and has potential value in automated arrhythmia classification.