Domain-adaptation fault diagnosis of mechanical transmission system based on multi-modal fusion of vibration signals and time-frequency images

The health status of key components in mechanical transmission systems is crucial to equipment reliability. Although transfer learning-based fault diagnosis methods improve model adaptability to different working conditions, existing methods mostly rely on single-modal data, while most multi-modal approaches focus on same-domain scenarios and existing multi-modal cross-domain methods remain relatively complex. This work proposes a novel lightweight unsupervised domain adaptation (UDA) method for fault diagnosis of multiple mechanical components, using single-sensor multi-modal representations, including vibration signals and their corresponding time-frequency images. Initially, a multi-modal data set integrating two-type data is built. A dual-branch lightweight convolutional neural network (CNN) extracts fault features from vibration signals and their time-frequency images with low parameters and computational overhead. Subsequently, contrastive learning (CL) enhances semantic consistency among multi-modal features. Then, multi-modal fusion facilitates complementary information exchange between the two kinds of feature representations. Finally, a conditional domain adversarial network (CDAN) aligns cross-domain distributions of fused features, boosting model generalization under diverse working conditions. Multiple cross-domain tasks covering varied speeds and loads, together with single and compound faults of bearings, rotors and gears are studied on a self-built data set. Results show that the proposed method achieves an average accuracy of 99.69% while maintaining low complexity and high inference speed. Additional validation on a public bearing benchmark data set achieves at least 99.04% accuracy across four transfer tasks. This demonstrates synergistic optimization of lightweight design, high accuracy, and strong generalization, providing an effective solution for fault diagnosis of multiple mechanical transmission components under diverse working conditions.