Thermal-Aware Station-Keeping Control of Stratospheric Airships Using Deep Reinforcement Learning

Long-endurance station-keeping control of stratospheric airships under atmospheric and thermal disturbances is investigated. To overcome the limitations of conventional three-degree-of-freedom (3-DOF) point-mass models, a 4-DOF dynamic model is developed by explicitly incorporating the yaw degree of freedom, which accounts for orientation-dependent aerodynamic drag. Integrated with a thermodynamic model and realistic stratospheric environmental profiles, the resulting simulation environment captures the combined aerodynamic and thermal influences on the vehicle dynamics, enabling finer-grained control responses to aerodynamic and thermal disturbances. A continuous control strategy based on proximal policy optimization is benchmarked against a discrete dueling double deep Q-network (D3QN) baseline. Under the coupled environment, the evaluated D3QN configuration exhibits larger seed-to-seed variability and more frequent early termination; the measured gap may reflect action quantization, algorithm-specific settings, and training stochasticity. The proposed thermal-aware approach achieves an average station-keeping error of [Formula: see text] while mitigating differential-pressure violation risks and retaining a higher energy reserve than the evaluated D3QN baseline. Seasonal tests further show that the May-trained policy transfers effectively to October, while the January and August trajectories reach the 30 km distance boundary. The framework provides a simulation-based approach for balancing station-keeping accuracy, thermal safety, and energy efficiency in autonomous stratospheric airship operations.