Sim-to-real UUV navigation in confined waters via unified sonar perception and incremental-reference reinforcement learning

Local navigation of unmanned underwater vehicles (UUVs) in confined waters is challenged by sparse sonar observations, partial observability, and limited reference trackability during sim-to-real deployment. This study develops a hierarchical navigation framework integrating a unified sparse sonar abstraction (USSA), a recurrent incremental soft actor-critic (RI-SAC) planner, and a constrained execution layer. USSA maps simulated and real sonar measurements into a shared sector-wise representation, while RI-SAC combines frame stacking and LSTM memory to generate bounded increments of surge-speed and yaw-rate references. A bounded surge-thrust mapping and yaw-rate Tube MPC are used for physical execution. Under the same interaction budget, RI-SAC outperformed proximal policy optimization (PPO), twin delayed deep deterministic policy gradient (TD3), and standard SAC in late-stage navigation performance. The complete configuration achieved a 96.1% success rate, a 3.6% collision rate, and a reference-smoothness index of 0.034. Compared with the recurrent direct-reference policy, the incremental-reference design increased the success rate by 3.3 percentage points and reduced the smoothness index by 68.8%. In controlled-tank experiments without online fine-tuning, the incremental-reference policy achieved a 91.25% success rate, compared with 75.0% for the direct-reference policy, 52.5% for the dynamic window approach (DWA), and 63.75% for the vector field histogram plus (VFH + ). These results demonstrate the effectiveness of the proposed framework for sim-to-real sonar-based UUV navigation in controlled confined-water environments.