A neuro-dynamic programming approach to admission control in ATM networks: the single link case

We are interested in solving large-scale Markov decision problems. The classical method of dynamic programming provides a mathematical framework for finding optimal solutions for a given Markov decision problem. However, dynamic programming algorithms become computationally infeasible when the underlying Markov decision problem evolves over a large state space. In recent years, a new methodology, called neuro-dynamic programming, has emerged which tries to overcome this "curse of dimensionality". We show how neuro-dynamic programming can be applied to the admission control problem for a single link in an ATM environment. Based on results obtained through neuro-dynamic programming, we derive a heuristic "threshold" policy. Performances of the policies obtained through neuro-dynamic programming are compared with a policy which always accepts a customer when the required resources are available.