A cross-modal recognition model for high-risk cardiac diseases based on time–frequency knowledge feature extraction and adaptive attention

Existing ECG classification methods have difficulty jointly representing time- and frequency-domain information and are limited in their ability to fuse multiple views, model long-range dependencies, and handle class imbalance. To address these limitations, this study proposes a Time-Frequency Knowledge Attention Cross-Modal Fusion Network (TFKACF-Net). First, raw ECG signals and their DWT sequences are used to construct a dual-branch input for extracting complementary time–frequency features. Then, residual convolution, frequency-domain enhancement, and channel attention are employed to strengthen the representation of key waveform patterns. Finally, Transformer-based self-attention dynamically fuses multi-view features and captures long-range dependencies. Focal loss and AdEMAMIX are further combined to improve minority-class recognition. On the MIT-BIH five-class task, TFKACF-Net achieves an accuracy of 99.38% and a mean F1 score of 95.34%; on the PTB-XL multi-label five-class task, it achieves a mean AUC of 92.65%. Ablation and visualization experiments verify the effectiveness of the key modules, demonstrating that TFKACF-Net provides an effective approach to intelligent ECG classification.