What question did this study set out to answer?

To improve cooperative game guidance for multiple aircraft under complex constraints such as no-fly zones.

April 16, 2026Aerospace1 citationsOpen Access

Multi-Agent Reinforcement Learning-Based Intelligent Game Guidance with Complex Constraint

FLFucong LiuPLA Rocket Force University of Engineering YGYang GuoPLA Rocket Force University of Engineering SWShaobo WangPLA Rocket Force University of Engineering

Key Points

To improve cooperative game guidance for multiple aircraft under complex constraints such as no-fly zones.
Developed TP-MADDPG algorithm for cooperative guidance with NFZ avoidance
Established a nonlinear kinematic model of the three-body game (Evader–Pursuer–Defender)
Proposed LSTM Actor–Critic framework for capturing patterns in adversarial scenarios
Introduced a priority-driven adaptive sampling mechanism to enhance efficiency and policy convergence
Formulated a reward function to address credit assignment and reward sparsity issues.
Significant improvement in cooperative game guidance performance
Enhanced ability to manage complex constraints during simulations
Superior performance compared to existing multi-agent reinforcement learning baselines

Abstract

For the complex problems of multi-aircraft cooperative game guidance with No-Fly Zone (NFZ) avoidance and cross-task constraint propagation, a deep deterministic policy gradient algorithm with temporal awareness and priority cooperative optimization (TP-MADDPG) is proposed. Based on the three-body cooperative guidance, a new coupled guidance task is formed by adding the NFZ avoidance constraint. At the same time, considering the constraint compatibility problem in dynamic task switching, the cooperative aircraft are modeled as independent agents with differentiated policy networks. First, a nonlinear kinematic model of the three-body game constructed by Evader–Pursuer–Defender is established. And four complex constraint conditions, namely homing guidance, NFZ avoidance, collision avoidance, and cooperative guidance, are modeled separately. Secondly, the Long Short-Term Memory-based (LSTM) Actor–Critic framework is proposed to dynamically capture the evolution patterns of adversarial scenarios by mining hidden correlations in historical state-action sequences. This enables smooth policy transitions between the cooperative guidance phase and subsequent homing guidance phase, effectively addressing the challenges of environmental non-stationarity and temporal task dependencies. Then, a priority-driven adaptive sampling mechanism is proposed along with a heterogeneous roles cooperative reward function to specifically address credit assignment imbalance and sparse reward problems, respectively. The sampling mechanism capitalizes on the efficient retrieval properties of SumTree data structures while integrating bias correction techniques to expedite policy gradient convergence. The reward function utilizes the reward shaping method to formulate cooperative reward components that explicitly capture behavioral correlations among agents. Finally, simulations show that the proposed method significantly outperforms multi-agent reinforcement learning baselines, effectively improving the performance of cooperative game guidance under complex constraints.

KI fragen

Bookmark

View Full Paper