USV Formation and Path-Following Control via Deep Reinforcement Learning With Random Braking

This article addresses the problem of path following for underactuated unmanned surface vessels (USVs) formation via a modified deep reinforcement learning with random braking (DRLRB). A formation control model based on deep reinforcement learning (DRL) is constructed to urge USVs to form a preset formation. Specifically, an efficient reward function is designed from the perspective of velocity and error distance of each USV related to the given formation, and then a novel random braking mechanism is formulated to prevent the training of the decision-making network from falling into the local optimum and failing to achieve the training objectives. Following that, a virtual leader-based path-following guidance system is developed for the USV formation problem. Wherein, with the aid of DRLRB, our proposed system can adjust formation automatically and flexibly even when some USVs deviate from the formation. Simulation verifies the effectiveness and superiority of our formation and path-following control strategy.