Open-set domain generalization residual feature reconstruction network for intelligent fault diagnosis under unknown time-varying working conditions

Currently, deep learning has garnered significant attention for addressing fault diagnosis challenges. However, in real-world industrial settings, the complexity of equipment and diversity of operating conditions can lead to unknown faults occurring under target conditions. Consequently, open-set domain generalization fault diagnosis (OSDGFD) has emerged as a critical research direction. However, existing open-set domain generalized fault diagnosis suffers from negative transfer, drastic changes in feature distributions under non-stationary time-varying rotational speed conditions, and insufficient revelation of fault features. These issues severely hinder performance research in open-set domain generalized fault diagnosis. Therefore, this paper proposes the Residual Feature Reconstruction Network for Intelligent Fault Diagnosis in Unknown Time-Varying Conditions (RFR-OSDG). First, a compact and reconstructable latent representation for known-class features is learned using known-class feature distribution mechanisms, while KL divergence constrains the normality of the latent distribution. Unknown classes, failing to conform to this distribution, exhibit larger reconstruction errors and are thus preliminarily distinguished. Building upon this foundation, an adaptive domain-aware class boundary loss constructs a dynamic decision boundary. This boundary constrains intra-class samples to remain within the boundary region while excluding out-of-class samples, thereby enhancing the model’s boundary recognition for known classes and reducing cross-class confusion and misclassification of unknown classes. By minimizing the triplet loss, the aggregation of similar features and the separation of dissimilar features are enhanced. An entropy-enhanced category-aware mechanism is designed to address the issue of inconsistent data distribution between the source domain and the target domain. Finally, experimental results under non-stationary time-varying speed show that the proposed method, RFR-OSDG, has better clustering and classification performance as well as superior generalization.