A Novel Approximate Dynamic Programming Structure for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

In this brief, for time-varying nonlinear systems, a novel approximate dynamic programming structure is first proposed to obtain the approximate optimal solution of the control system. In addition, considering that the system is time-varying and it is difficult to obtain the system dynamics, we propose the idea of time normalization to identify the time-varying system dynamics through a model network. Based on the traditional heuristic dynamic programming structure, an action network is added to approximate the next time step action. With the increase of feedback channels, the convergence speed of the action network is accelerated. And we also prove that the weight errors of the three improved neural network structure are uniformly ultimately bounded. The effectiveness of our proposed new neural network structure is demonstrated by numerical simulation results.