Neuro-dynamic programming: an overview

We discuss a relatively new class of dynamic programming methods for control and sequential decision making under uncertainty. These methods have the potential of dealing with problems that for a long time were thought to be intractable due to either a large state space or the lack of an accurate model. The methods discussed combine ideas from the fields of neural networks, artificial intelligence, cognitive science, simulation, and approximation theory. We delineate the major conceptual issues, survey a number of recent developments, describe some computational experience, and address a number of open questions.