Research on Collision Avoidance Algorithm of Unmanned Surface Vehicle Based on Deep Reinforcement Learning

To solve the problem of active collision avoidance for unmanned surface vehicles (USVs) in a complex maritime environment, a method of deep reinforcement learning (DRL) based on proximal policy optimization (PPO) is proposed. To master the collision avoidance policy by self-learning, the mathematical model of USV, dynamic obstacle generation model, and reward mechanism were established. Using the relative position between the obstacle and USV and the information of the closest point of approach (CPA), high-dimensional state features including the collision track layer and collision threat layer were constructed. On this basis, combined with low-dimensional states such as navigation state and path error, a deep convolutional neural network (CNN) structure fused in multifeature and multiscale was designed. The proposed DRL network was trained through repeated collision avoidance simulations in random environments. Simulation results reveal that the proposed algorithm can output effective decisions in complex scenarios and avoid dynamic obstacles quickly and safely.