A Stochastic Approximation Algorithm with Varying Bounds

Many optimization problems that are intractable with conventional approaches will yield to stochastic approximation algorithms. This is because these algorithms can be used to optimize functions that cannot be evaluated analytically, but have to be estimated (for instance, through simulation) or measured. Thus, stochastic approximation algorithms can be used for optimization in simulation. Unfortunately, the classical stochastic approximation algorithm sometimes diverges because of unboundedness problems. We study the convergence of a variant of stochastic approximation defined over a growing sequence of compact sets. We show that this variant converges under more general conditions on the objective function than the classical algorithm, while maintaining the same asymptotic convergence rate. We also present empirical evidence that shows that this algorithm sometimes converges much faster than the classical algorithm.