Optimization of Path Following for Autonomous Surface Vehicles Using Reinforcement and Imitation Learning

Abstract Path following in marine vessels presents significant challenges due to complex wave dynamics. This paper addresses both path-following accuracy and fuel consumption optimization through reduced rudder fluctuations. A novel multi-objective Pareto optimization approach is introduced to determine optimal reward function weights for deep reinforcement learning, replacing conventional trial-and-error methods. The study presents a hybrid learning framework that integrates Generative Adversarial Imitation Learning (GAIL) with Proximal Policy Optimization (PPO), combining PPO’s exploration capabilities with GAIL’s demonstration learning. Expert demonstrations are generated using PD controller-based MMG dynamics in wave conditions, while the reinforcement learning environment utilizes Nomoto dynamics in calm water. Comparative analysis against standalone PPO and GAIL implementations across various path-following scenarios demonstrates that the hybrid policy achieves comparable path-following accuracy while significantly reducing the rudder actuation. The results indicate that this hybrid approach is particularly effective for real-world applications where control surface efficiency is crucial and high-quality expert demonstrations are limited.