A DRL Framework for Autonomous Pursuit-Evasion: From Multi-Spacecraft to Multi-Drone Scenarios

To address the challenges of autonomous pursuit-evasion in aerospace, particularly in achieving cross-domain generalizability and handling complex terminal constraints, this paper proposes a generalizable deep reinforcement learning (DRL) framework. The core of the method is a self-play Proximal Policy Optimization (PPO) architecture enhanced by two key innovations. First, a dynamics-agnostic curriculum learning (CL) strategy is employed to accelerate training and enhance policy robustness by structuring the learning process from simple to complex. Second, a transferable prediction-based reward function is designed to provide dense, forward-looking guidance, utilizing forward-state projection to effectively satisfy mission-specific terminal conditions. Comprehensive simulations were conducted in both multi-spacecraft and multi-drone scenarios. In the primary spacecraft validation, the proposed method achieved a 90.7% success rate, significantly outperforming baseline algorithms like traditional PPO and Soft Actor-Critic (SAC). Furthermore, it demonstrated superior robustness, with a performance drop of only 8.3% under stochastic perturbations, a stark contrast to the over 18% degradation seen in baseline methods. The successful application in a multi-drone scenario, including an obstacle-rich environment, confirms the framework’s potential as a unified and robust solution for diverse autonomous adversarial systems.