Optimal evasive maneuver planning for submarines against multi-torpedo threats using reinforcement learning

We propose a methodology for submarine evasion against multi-torpedo threats, extending conventional single-torpedo, geometry-based strategies through the use of reinforcement learning. Recent advancements in marine weaponry have shifted the threat paradigm from single-torpedo launches to simultaneous multi-torpedo attacks, exposing the limitations of existing geometry-based approaches. To address the lack of standardized multi-torpedo evasion tactics, we employ the proximal policy optimization (PPO) algorithm to determine optimal evasive maneuvers in three-dimensional combat scenarios. Our approach models submarine motion dynamics (e.g., turn radius, acceleration, velocity limits), decoy deployment, torpedo pursuit behavior, and sonar systems. By systematically exploring the continuous state space—including submarine positions, headings, and velocities—our PPO-based agent learns adaptive strategies that balance rapid course changes, effectively evading multi-torpedo engagements. We validate the proposed method via MATLAB simulations, demonstrating that the PPO-driven policy outperforms traditional geometry-based approaches in terms of both times to intercept and overall survival probability. This work was conducted with research funding support from LIG Nex1.