Research on Domain Generalization Methods Based on Attention Mechanism

Introduction/Objective: Domain generalization (DG) has emerged as a prominent research focus in deep learning, yet few existing works have investigated attention mechanisms in DG. This study explores how attention mechanisms work for domain generalization and leverages optimized attention modules to improve model cross-domain generalization performance. Methods: This study proposes attention-driven domain generalization approaches: (1) Attention Masking Module (AMM), which masks features with high attention weights, is simple to deploy with low computational overhead for real-world applications; (2) Attention-Based Representation Feature Decoupling (ABRFD), which achieves accurate separation of domainspecific and domain-invariant features through attention-guided feature decoupling, focusing on delivering superior cross-domain generalization performance. Results: Both methods were evaluated on public benchmark datasets (PACS, VLCS, Office- Home, DomainNet). The experimental results show that both proposed methods achieve competitive generalization performance compared with state-of-the-art domain generalization methods. Compared with other regularization methods, the attention-driven framework achieves stronger generalization performance, Discussion: Our methods provide a feasible pathway to leverage attention to improve generalization performance during training, show that suppressing domain-specific shortcut features is more effective than enhancing only general features, and demonstrate that feature screening based on learned attention weights is an effective direction for DG development. Conclusion: This work verifies the dual role of attention mechanisms in domain generalization and proposes two practical attention-driven methods for DG tasks. Future work will focus on eliminating manual hyperparameter tuning, designing for lower computational cost, and expanding applicability in complex, real-world, cross-domain scenarios.