Robust reinforcement learning control with static and dynamic stability

Robust control theory is used to design stable controllers in the presence of uncertainties. This provides powerful closed‐loop robustness guarantees, but can result in controllers that are conservative with regard to performance. Here we present an approach to learning a better controller through observing actual controlled behaviour. A neural network is placed in parallel with the robust controller and is trained through reinforcement learning to optimize performance over time. By analysing nonlinear and time‐varying aspects of a neural network via uncertainty models, a robust reinforcement learning procedure results that is guaranteed to remain stable even as the neural network is being trained. The behaviour of this procedure is demonstrated and analysed on two control tasks. Results show that at intermediate stages the system without robust constraints goes through a period of unstable behaviour that is avoided when the robust constraints are included.