A COLREGs compliance reinforcement learning approach for USV manoeuvring in track-following and collision avoidance problems

The development of new technologies for autonomous platforms has allowed their integration into sea mine countermeasures. This has allowed to remove the personnel from the potential danger by having the mine search task performed by an unmanned surface vessel (USV). Traditional intelligent systems are built by agglomerating hand-coded behaviours that determine how a good manoeuvre looks like. This induces cognitive bias into the pre-defined behaviours that can violate safety and regulatory rules imposed by the COLREGs. To alleviate this issue, this paper proposes a COLREGs compliant reinforcement learning (RL) approach that gives a solution for the autonomous navigation of USVs. A custom simulation environment is developed. The RL agents are trained to deal with path-following problem with obstacle avoidance capabilities. A custom reward function is defined to consider the turning disks for the agent’s decision process. A smoothing decision feature is used to smooth the transitions between consecutive actions. The results demonstrate good convergence and high performance under different scenarios. The collision avoidance with COLREGs compliances shows the effectiveness of the proposed approach under several scenarios with static and moving obstacles. • A novel reward function design that complies with COLREGs’ regulations. • A decision-making mechanism for the smooth transition between actions in the USV. • An implementation that considers the rate of turn and orientation angle as features.