A physics-informed reinforcement learning framework for impulsive orbital pursuit–evasion under stochastic maneuvers

Autonomous orbital pursuit–evasion represents a complex differential game with impulsive dynamics and stochastic adversarial maneuvers. Although model-free reinforcement learning has shown promise in high-dimensional decision-making, it often suffers from slow convergence and instability, mainly due to high-variance value estimation caused by unobservable evader maneuvers. This paper proposes a Physics-Informed Proximal Policy Optimization (PI-PPO) framework featuring a dual-correction closed-loop mechanism. A Kalman-driven opponent-modeling module dynamically estimates impulsive maneuvers, enabling accurate physical prediction under maneuvering uncertainty and correcting the systematic prediction bias of the orbital dynamics model. Furthermore, a physics-consistency loss derived from the Clohessy–Wiltshire equations is incorporated into the critic to regularize value learning and promote physical coherence. These two mechanisms establish a closed loop where data corrects physics and physics constrains learning, ensuring lower-variance and more stable policy updates. Simulation results in impulsive orbital pursuit scenarios demonstrate that the proposed method achieves faster convergence, higher sample efficiency, and stronger robustness against maneuver uncertainty compared with baseline algorithms.