HJB Equation and Least Square Transform Based Optimal Control Design With Convergence Analysis

This paper researches the optimal control design of a linear system via an integral objective and the assumed Hamiltonian. By analyzing the temporal convergence of a value function, which is formulated as the Lyapunov function in stability transform, the iterative solution of Hamiltonian is proven to be suboptimal in approximating the objective. Then based on the discretization of the objective, the variant of the Hamilton‐Jacobi–Bellman (HJB) equation is derived to solve the optimal control. After utilizing the integral transform, an iterative HJB equation is obtained to seek the minimum objective, with the optimal control derived in virtue of neural network (NN) approximation and least square (LS) integration. Furthermore, the Riccatti equation in the case of a quadratic‐form objective is transformed from the HJB equation, with the complexity of the control system contrasted with the LS‐solved one, meanwhile, the coefficient matrix of the control law is sought and formulated from the proposed algorithm. Finally, the theoretical results are validated in numerical computation to show the refinement of the dynamic performance index by optimal control. It proves the stability of the iteration process with inequality transform, as well as the prospect of methods to be applied with regard to normal methods.