Synthetic Learning and Control: MAPPO-Tuned MAADRC with Graph-Laplacian Enhancement for Resilient Multi-USV Formation in Dynamic Maritime Settings

Formation control of unmanned surface vehicles (USVs) in complex marine environments is required to contend with strongly coupled, high-dimensional disturbances. A Multi-Agent Active Disturbance Rejection Control (MAADRC) framework is developed for this purpose. The design centers on a distributed extended state observer (DESO) coupled with a dual-channel feedback structure—NEFL-GCO and LGL-FC—that collectively maintains formation geometry. Three main ideas underpin the approach. First, a bandwidth-efficient distributed observation scheme enables agents to share disturbance estimates while using substantially less communication bandwidth. Second, an adaptive consensus compensation mechanism accommodates parameter variations as formations evolve. Third, a formation-compatible obstacle avoidance algorithm enhances reliability in congested waters. To evaluate the control structure and optimize its parameters, a multi-agent reinforcement learning (MARL) method—specifically Multi-Agent Proximal Policy Optimization (MAPPO)—is employed. The MARL agent tunes two critical parameters: observer bandwidth and nonlinear feedback gain, thereby establishing a performance baseline. After ten million training steps, the MAPPO-optimized MAADRC achieves a tracking root-mean-square error (RMSE) of 1.18 m. This value lies within 3% of the manually tuned result of 1.21 m, indicating that the bandwidth parameterization is near-optimal. Extensive simulations incorporating realistic wind, wave and current disturbances demonstrate a dynamic obstacle avoidance success rate maintaining an expected level, alongside consistently low formation tracking errors. Collectively, these findings confirm the resilience and practical utility of the proposed framework in demanding maritime settings.