Attention-Guided and Role-Aware Reinforcement Learning for Multi-AUV Counter-Game

This article proposes an attention-guided, role-aware multiagent deep reinforcement learning (MADRL) scheme to enhance collaborative decision-making among autonomous underwater vehicles (AUVs) in the counter-game (CG). First, a customized multi-AUV CG model is developed, incorporating AUV-specific constraints. Ashared reward alignment mechanism is introduced to synchronize individual and team objectives, promoting cooperation and enabling the emergence of heterogeneous behaviors. Second, a hybrid decision architecture integrating soft graph attention, recurrent structures, and contrastive role encoding is proposed to enhance AUV tactical cognition and context-aware responsiveness, while ensuring extension and adaptability. Finally, building on reward-guided optimization, a role-driven contrastive learning objective is introduced to maximize mutual information between individual behaviors and role representations, fostering heterogeneous coordination and diverse policy development. Comparative studies and multiscale games confirm the scheme's superiority in adaptability and coordination over existing baselines. The learned policies exhibit heterogeneous behaviors, such as focused fire and decoy tactics, which improve mission efficiency and safety. Model performance analysis and lake trials further validate its applicability. The related experimental videos are available at https://sjtu-mirus.github.io/MIRUS.github.io/research/counter-game.