Neural network control for mitigating actuator delay in ATR engines using predictive compensation and stability reward

The flight envelope of Air Turbo Rocket (ATR) engines is broader compared to conventional aero-engines, and designing a full-envelope controller using traditional methods poses significant challenges due to a burdensome design process. To address this issue, this paper proposes a self-learning neural network controller design method based on Reinforcement Learning (RL). Additionally, a method for predictive compensation and stability rewards is proposed to reduce the system oscillation caused by actuator delay. This approach simplifies the actuator to a first-order inertial element exhibiting pure delay. A simulation environment for the ATR engine-actuator system is first established. Based on this environment, a self-learning neural network controller using a predictive compensator and the Proximal Policy Optimization (PPO) algorithm is then developed. Furthermore, the temporal difference signals from the controller output are integrated into the reward function to enhance system stability. The proposed method is validated through numerical simulations and semi-physical experiments. The numerical simulation results demonstrate that the proposed method increases the system’s tolerance to delays from 20 ms to 400 ms. Under an actuator delay of 400 ms, the average steady-state error remains less than 0.1%, the overshoot is limited to 1%, and the settling time does not exceed 3 s. Moreover, compared to the traditional method, the proposed method exhibits higher adaptability to model errors and variations in flight conditions. In the conducted semi-physical simulation experiments, the proposed method achieves stable control of a real electric pump.