A Contrastive Multi-Modal Time-Frequency Swin Transformer Network for Semi-Supervised Mechanical Equipment Fault Diagnosis

Traditional deep learning networks in fault diagnosis tasks struggle to effectively utilize unlabeled data, and their diagnostic performance is constrained by the limited availability of labeled samples. To address this issue, this paper proposes a semi-supervised method based on multi-modal time-frequency fusion and an improved Swin Transformer, termed the Contrastive Multi-modal Time-Frequency Swin Transformer for Semi-Supervised Fault Diagnosis (CMTFST-SFD). First, a multi-scale shift window attention fusion module is designed as the backbone network, performing parallel computations with a 7 × 7 fine-grained window and a 14 × 14 coarse-grained window. Subsequently, a three-channel time-frequency encoder and a multi-frequency fusion convolution module are constructed to extract diverse frequency characteristics. Finally, by integrating a joint contrastive-cross-entropy loss function, the network backbone is trained in the first stage using a combination of unlabeled and labeled data, followed by training the classification head solely with labeled data in the second stage, thereby fully leveraging all available data for comprehensive model training. The proposed method is evaluated on two bearing fault datasets under three different labeling rates, consistently achieving a recognition rate exceeding 97%. Compared to other advanced techniques, the proposed method demonstrates superior recognition rates and enhanced clustering performance.