Deep Reinforcement Learning-based Navigation of Unmanned Surface Vessel in Unknown Environments

Current deep reinforcement learning (DRL)-based path planning methods for Unmanned Surface Vessels (USVs) face limitations in unknown dynamic marine environments, such as insufficient adaptation to dynamic ocean conditions, redundant image feature interference, and ineffective handling of dynamic trajectory information, which restrict their generalization performance and navigation safety. To address these issues, this paper proposes a novel DRL-based path planning method to enhance the safety and stability of USV autonomous navigation in unknown waters. The core of the method is a Dynamic Trajectory Temporal-Convolution Neural Network (DTT-CNN), which is designed with an improved structure integrating the Atrous Spatial Pyramid Pooling (ASPP) module and Long Short-Term Memory (LSTM) module to process dynamic marine environment information, reduce redundant image features, and effectively handle dynamic trajectory data. An ocean current disturbance model is introduced to improve the algorithm’s generalization in practical scenarios, and the state space, action space, and composite reward function of the algorithm are carefully designed to ensure safe and accurate navigation to the destination. Comprehensive validation experiments and comparative analyses are conducted against traditional SAC, ASPP-SAC, LSTM-SAC, and SDDQN algorithms in simulated environments with static obstacles, dynamic obstacles, and random obstacles. The results demonstrate that the proposed method achieves higher navigation success rate (over 0.9 after 5000 training episodes), shorter average path length, and more stable navigation performance compared to baseline algorithms. This study innovatively proposes the DTT-CNN structure that balances the processing of ocean current disturbances, dynamic obstacle trajectories, and navigation objectives, solving the problem of poor integration of temporal information and redundant feature interference in existing DRL-based methods, and providing a more effective and robust path planning solution for USV navigation in unknown dynamic marine environments.