UAV formation control for multi-target tracking based on an improved multi-agent double soft actor–critic

In recent years, formation control strategies based on multi-agent deep reinforcement learning (MADRL) have shown great potential for coordinating unmanned aerial vehicles (UAVs) in multi-target tracking (MTT) tasks. However, in complex, high-dimensional decision-making environments, the value evaluation networks in MADRL are prone to overestimation bias, undermining the stability of policy learning. Meanwhile, the risk of collisions among UAVs and uneven spatial distribution further limit the coordination efficiency and tracking performance of multi-UAV systems. To address these challenges, an improved multi-agent double soft actor–critic (i-MADSAC) framework is proposed for cooperative MTT tasks. Specifically, a centralized double-critic network based on an attention mechanism is developed to reduce variance and adapt to varying input dimensions by optimizing the double-critic network. Then, a distance-based inverse-square repulsion function is designed to effectively mitigate collision risks. In addition, a spatial information-entropy-based reward function is designed to optimize the spatial distribution and coverage of UAVs. It works in conjunction with the repulsion function to ensure efficient collision avoidance. Experimental results demonstrate that the proposed i-MADSAC exhibits superior robustness, scalability, and tracking performance as the number of UAVs and task complexity increase, providing an efficient and reliable solution for MTT applications.