Currently, unmanned aerial vehicle (UAV) formation flight control still relies on model-based information, and the level of UAV intelligent control is relatively low. To address this issue, a deep reinforcement learning approach is investigated. Firstly, corresponding reinforcement learning elements are devised for formation control problems, and formation controllers based on the deep reinforcement learning Deep Q-Network (DQN) algorithm are designed. Simultaneously, a method integrating a priority strategy with a multi-layer action library is proposed to accelerate the algorithm convergence and enable the wingman to ultimately maintain the expected range. Finally, the designed controller is compared with the Proportional-Integral-Derivative (PID) controller through simulation, and the effectiveness of the DQN controller is verified. The simulation results indicate that the controller can be applied to UAV formation, enhance the intelligence of the wingman, maintain the expected distance through autonomous learning, and the controller design does not require accurate model information, which provides a basis and reference for the intelligent control of UAV formation.
Zhenqi He (Fri,) studied this question.