The Relationship between the Maximum Principle and Dynamic Programming

Let $V(t,x)$ be the infimum cost of an optimal control problem, viewed as a function of the initial time and state $(t,x)$. Dynamic Programming is concerned with the properties of V( · , · ) and in particular with its characterization as a solution to the Hamilton–Jacobi–Bellman equation. Heuristic arguments have long been advanced relating the Maximum Principle to Dynamic Programming according to \[p(t) = - V_x ( {t,x_0 (t)} ).\] Here x₀ ( · ) is the minimizing state function under consideration and p( · ) is the costate function of the Maximum Principle. In this paper we examine the validity of such claims and find that this relationship, interpreted as a differential inclusion involving the generalized gradient, is indeed true, almost everywhere and at the endpoints, for a very large class of nonsmooth optimal control problems.