A Proximal Policy Optimization method in UAV swarm formation control

UAV swarms have increasingly replaced human labor in various industries. However, controlling a large group of UAVs can be difficult and, if not done correctly, cause significant financial losses and delays. While many methods aim to improve how the group is controlled, little focus is given to optimizing and stabilizing movement trajectories, which limits the emergency response and reliability of UAV swarm in dynamic environments. This paper proposes a new method to control UAVs through Proximal Policy Optimization. This method utilizes two neural networks and incorporates the concept of a game. During training, one neural network generates an action based on the current state, while the other evaluates the output of the first network. With continuous refinement, the performance of both networks can be enhanced, ultimately leading to the optimal decision-making model. Additionally, this research introduces a hierarchical management mechanism to address the issue of complex computations in large bee colonies and to distribute control more evenly. Simulation results demonstrate that this approach can successfully reconstruct formations under various scenarios. Compared to similar algorithms, this method is at the forefront of tackling large-scale problems, with a collision rate close to 0 and a 100% success rate in emergency processing.