Collision Avoidance of Multi-UUV Systems Based on Deep Reinforcement Learning in Complex Marine Environments

For multiple unmanned underwater vehicles (UUVs) systems, obstacle avoidance during cooperative operation in complex marine environments remains a challenging issue. Recent studies demonstrate the effectiveness of deep reinforcement learning (DRL) for obstacle avoidance in unknown marine environments. However, existing methods struggle in marine environments with complex non-convex obstacles, especially during multi-UUV cooperative operation, as they typically simplify environmental obstacles to convex shapes with sparse distributions and ignore the dynamic coupling between cooperative operation and collision avoidance. To address these limitations, we propose a centralized training with decentralized execution framework with a novel multi-agent dynamic encoder based on an efficient self-attention mechanism. The framework, to our knowledge, is the first to dynamically process observations from an arbitrary number of neighbors that effectively addresses multi-UUV collision avoidance in marine environments with complex non-convex obstacles while satisfying additional constraints derived from cooperative operation. Experimental results show that the proposed method effectively avoids obstacles and satisfies cooperative constraints in both simulated and real-world scenarios with complex non-convex obstacles. Our method outperforms typical collision avoidance baselines and enables policy transfer from simulation to real-world scenarios without additional training, demonstrating practical application potential.