Extended adaptive optimal control of linear systems with unknown dynamics using adaptive dynamic programming

The extended infinite horizon optimal control problem of continuous time linear systems with unknown dynamics is investigated in this paper. This optimal control problem can be solved using the corresponding extended algebraic Riccati equation. A new policy iteration algorithm is proposed to approximate the solution of the extended algebraic Riccati equation when the system dynamics are known. The convergence of the proposed algorithm is proved. Based on the proposed policy iteration algorithm, an online adaptive dynamic programming (ADP) algorithm is developed to find the solution to the extended infinite horizon optimal control problem of unknown continuous time linear systems. The convergence of the online ADP algorithm is analyzed. Finally, two simulation examples are given to demonstrate the effectiveness of the developed online ADP algorithm.