TAP-DDQN: Multiplicative Potential-Based Reward Shaping Framework for Tactical Decision-Making of Unmanned Surface Vehicles in Adversarial Maritime Engagements

Unmanned Surface Vehicles (USVs) increasingly require on-board policies capable of engagement-level tactical decisions under discrete mission-system constraints. Most existing systems, however, remain rule-based and predictable, and many maritime reinforcement learning studies still focus on low-level continuous control. This paper proposes TAP-DDQN, a GRU-enhanced dueling deep Q-network trained with a multiplicative Tactical Approach Potential (TAP) and a dynamic target curriculum for discrete tactical decision-making in adversarial maritime engagements. The proposed TAP couples a desired engagement-range ring with an aspect-aware tactical term so that angular shaping becomes active mainly inside the tactical band, providing dense guidance without encouraging irrelevant long-range aspect optimization. The method is implemented through the non-invasive TRNLE wrapper, which enables learning and deployment without modifying the legacy combat-management software stack. In a 1-vs-1 surface-engagement scenario, the proposed agent achieves a DGAR of 54.87±4.19% and consistently outperforms feedforward and non-TAP baselines on reward-aligned diagnostics and maneuver consistency; relative to the strongest learning baseline (DQN+MLP without TAP), the improvement is 8.13 percentage points. Qualitative analyses further show that the learned policy approaches the desired engagement ring, stabilizes a favorable stern-quarter geometry, and exhibits more coherent maneuver behavior than rule-based or additive-reward baselines. These results support multiplicative TAP shaping as an effective and deployment-compatible approach for USV tactical autonomy and intelligent adversary generation in naval training environments.