Hierarchical Decision Making-Based Intelligent Game Confrontation on UAV Swarm

To address the challenge of decision making in close-range air combat for fixed-wing unmanned air vehicle (UAV) swarms, this paper proposes a distributed Hierarchical Cooperative Soft Actor-Critic with maximum entropy (HC-SAC) framework. A top-level target decision-making and bottom-level maneuvering framework is designed to resolve convergence issues in traditional multi-agent reinforcement learning, typically for long mission durations and complex state spaces. Friendly tactics are incorporated into top-level decisions to enhance coordination, with both offensive and defensive sub-policies balanced for swarm confrontations. These policies are trained using the Soft Actor-Critic (SAC) deep reinforcement learning algorithm with specific reward functions, and the effectiveness of this method is verified through 5v5 swarm game confrontation simulations.