A Reinforcement Learning-Based Approach for Optimal Output Tracking in Uncertain Nonlinear Systems with Mismatched Disturbances

In this paper, the optimal control problem of uncertain nonlinear systems is considered. A nonlinear disturbance observer (NDO) is proposed to measure the lumped uncertainties present in the system. Disturbances that do not enter the same channel as the control signal, so-called mismatched disturbances, are difficult to reject directly within the control channel. To overcome the challenge, a generalized disturbance observer-based compensator is implemented to address the uncertainty compensation problem by attenuating its influence on the output channel. In real time, by augmenting the system states with the output tracking error, we develop a composite actor-critic reinforcement learning (RL) scheme for approximating the optimal control policy as well as the ideal value function pertaining to the compensated system by solving the Hamilton-Jacobi-Bellman (HJB) equation. Concurrent learning is applied in this article by using the recorded data of the known model of the system, in order to enhance the robustness of the system by canceling the influence of the probing signal. Simulation results demonstrate the effectiveness of the proposed scheme, offering an optimal solution for the output tracking problem in a second-order model with mismatched disturbances.