Optimal Controller Synthesis Using Approximating-Graph Dynamic Programming

Dynamic programming is well known as a method of calculating optimal control but is not often used in practice because it is assumed to be computationally expensive. We introduce a new stageless version of dynamic programming that produces numerical approximations to optimal control laws for continuous systems. The method creates an approximating graph that models the possible state transitions in the continuous system. Dynamic programming is applied to the approximating graph to obtain a control table. Forward dynamic programming is the exact dual of reverse dynamic programming in this formulation. The approach is evaluated for three problems, including two fourth order systems.