Stratospheric airship altitude control based on thermal coupling effects and deep reinforcement learning

To investigate the altitude control issue of stratospheric airships, a dynamical model and a thermodynamic model of stratospheric airships were established. The Double Deep Q-Network (DDQN) algorithm, which introduces target networks and dual Q-value estimation to reduce the overestimation of Q-values and thereby enhances the performance of the DQN algorithm, was employed. The idea of artificial potential field method was introduced to improve the network convergence speed of DDQN algorithm. Taking the on/off state of the valve as the action input, the improved algorithm is applied to the altitude control of stratospheric airships and analyzed in comparison with the effect of fuzzy PID control. Simulation results show that the DDQN algorithm has significant advantages over the fuzzy PID algorithm in terms of response time, overshoot, and oscillation during the altitude adjustment process, with a steady-state error within 20 meters. Under the consideration of uncertainties, DDQN exhibits better robustness and convergence efficiency.