Artificial Intelligence for Cooperative Collision Avoidance of Ships Developed by Multi-Agent Deep Reinforcement Learning

Abstract Most ship collisions are caused by human error, making them unpreventable as long as humans operate a ship. Autonomous navigation technology is expected to prevent these accidents. Collision avoidance is a key issue in the realization of autonomous ships. To date, rule- and artificial intelligence (AI)-based collision avoidance algorithms have been studied. However, most of them assumed a simple environment in which other ships maintain their speed and course. During actual navigation in congested waters, safe and efficient collision avoidance are achieved through cooperative maneuvers between ships that obey the COLREGs. To realize cooperative maneuvers of ships, multi-agent deep reinforcement learning was applied to learn collision avoidance maneuvers considering the motions of other ships and COLREGs. Then, the developed multi-agent AI was evaluated with typical one-on-one encounters. The multi-agent AI performed cooperative collision avoidance maneuvers in head-on and crossing situations. Furthermore, the influence of reward setting regarding the degree of compliance with COLREGs on the cooperative collision avoidance was investigated. These results demonstrated that the multi-agent AI was promising for realizing cooperative collision avoidance maneuvers similar to those performed by humans.