Off-Policy Deep Reinforcement Learning for Path Planning of Stratospheric Airship

The stratospheric airship is a vital platform in near-space applications, and achieving autonomous transfer has become a key research focus to meet the demands of diverse mission scenarios. The core challenge lies in planning feasible and efficient paths, which is difficult for traditional algorithms due to the time-varying environment and the highly coupled multi-system dynamics of the airship. This study proposes a deep reinforcement learning algorithm, termed reward-prioritized Long Short-Term Memory Twin Delayed Deep Deterministic Policy Gradient (RPL-TD3). The method incorporates an LSTM network to effectively capture the influence of historical states on current decision-making, thereby improving performance in tasks with strong temporal dependencies. Furthermore, to address the slow convergence commonly seen in off-policy methods, a reward-prioritized experience replay mechanism is introduced. This mechanism stores and replays experiences in the form of sequential data chains, labels them with sequence-level rewards, and prioritizes high-value experiences during training to accelerate convergence. Comparative experiments with other algorithms indicate that, under the same computational resources, RPL-TD3 improves convergence speed by 62.5% compared to the baseline algorithm without the reward-prioritized experience replay mechanism. In both simulation and generalization experiments, the proposed method is capable of planning feasible paths under kinematic and energy constraints. Compared with peer algorithms, it achieves the shortest flight time while maintaining a relatively high level of average residual energy.