Image Enhancement Based on Dual-Branch Generative Adversarial Network Combining Spatial and Frequency Domain Information for Imbalanced Fault Diagnosis of Rolling Bearing

To address the problems of existing 2D image-based imbalanced fault diagnosis methods for rolling bearings, which generate images with inadequate texture details and color degradation, this paper proposes a novel image enhancement model based on a dual-branch generative adversarial network (GAN) combining spatial and frequency domain information for an imbalanced fault diagnosis of rolling bearing. Firstly, the original vibration signals are converted into 2D time–frequency (TF) images by a continuous wavelet transform, and a dual-branch GAN model with a symmetric structure is constructed. One branch utilizes an auxiliary classification GAN (ACGAN) to process the spatial information of the TF images, while the other employs a GAN with a frequency generator and a frequency discriminator to handle the frequency information of the input images after a fast Fourier transform. Then, a shuffle attention (SA) module based on an attention mechanism is integrated into the proposed model to improve the network’s expression ability and reduce the computational burden. Simultaneously, mean square error (MSE) is integrated into the loss functions of both generators to enhance the consistency of frequency information for the generated images. Additionally, a Wasserstein distance and gradient penalty are also incorporated into the losses of the two discriminators to prevent gradient vanishing and mode collapse. Under the supervision of the frequency WGAN-GP branch, an ACWGAN-GP can generate high-quality fault samples to balance the dataset. Finally, the balanced dataset is utilized to train the auxiliary classifier to achieve fault diagnosis. The effectiveness of the proposed method is validated by two rolling bearing datasets. When the imbalanced ratios of the four datasets are 0.5, 0.2, 0.1, and 0.05, respectively, their average classification accuracy reaches 99.35% on the CWRU bearing dataset. Meanwhile, the average classification accuracy reaches 96.62% on the MFS bearing dataset.