Autonomous combat unmanned aerial vehicle (UAV) systems represent a critical area of research for global military powers. This study investigates one-on-one close-range air combat scenarios involving UAVs and proposes an autonomous maneuvering decision-making model based on deep reinforcement learning (DRL). A six-degree-of-freedom continuous action space is established to simulate UAV combat environments realistically. In the proposed decision-making model, a global reward function is primarily designed based on the combat outcome and a guidance reward that incorporates four key tactical dimensions: attack angle, distance, velocity, and altitude. In addition, we design a value-based prioritized experience replay (PER) mechanism to improve sample efficiency by adaptively balancing old and new experiences, thereby accelerating convergence. Finally, a three-dimensional air combat simulation environment is developed. Experimental results demonstrate that the proposed model achieves strong convergence and practical effectiveness in autonomous air combat decision-making, significantly outperforming baseline methods in terms of tactical adaptability and mission success rates.
Liu et al. (Mon,) studied this question.