Loyal wingman task execution for future aerial combat: A hierarchical prior-based reinforcement learning approach

In modern Beyond-Visual-Range (BVR) aerial combat, unmanned loyal wingmen are pivotal, yet their autonomous capabilities are limited. Our study introduces an advanced control algorithm based on hierarchical reinforcement learning to enhance these capabilities for critical missions like target search, positioning, and relay guidance. Structured on a dual-layer model, the algorithm's lower layer manages basic aircraft maneuvers for optimal flight, while the upper layer processes battlefield dynamics, issuing precise navigational commands. This approach enables accurate navigation and effective reconnaissance for lead aircraft. Notably, our Hierarchical Prior-augmented Proximal Policy Optimization (HPE-PPO) algorithm employs a prior-based training, prior-free execution method, accelerating target positioning training and ensuring robust target reacquisition. This paper also improves missile relay guidance and promotes the effective guidance. By integrating this system with a human-piloted lead aircraft, this paper proposes a potent solution for cooperative aerial warfare. Rigorous experiments demonstrate enhanced survivability and efficiency of loyal wingmen, marking a significant contribution to Unmanned Aerial Vehicles (UAV) formation control research. This advancement is poised to drive substantial interest and progress in the related technological fields.