This paper reviews the application of adaptive dynamic programming (ADP) in controlling constrained nonlinear systems, with an emphasis on integrating ADP to address optimization and constraint issues in complex systems. First, the conventional ADP algorithm for tackling the optimal control problems of unconstrained nonlinear systems is introduced. Next, the general solutions and recent advances of ADP for controlling nonlinear systems with various constraints, mainly including input constraints, state constraints, output constraints and cost constraints, are elaborated. Moreover, several typical real control applications for constrained nonlinear systems with respect to ADP are summarized, particularly in the fields of aerospace systems, robots, autonomous systems and energy systems. Finally, some possible future prospects are explored, and the conclusions of this paper are presented. Overall, ADP plays a significant role in guaranteeing system safety and optimizing performance with broad application prospects, and the comprehensive investigation demonstrates the tremendous potential of ADP for controlling constrained nonlinear systems in the current eras of artificial intelligence, control science and engineering, and systems science.
Zhao et al. (Sat,) studied this question.