Convergence of parameter sensitivity estimates in a stochastic experiment

To reduce the error in estimating the gradient (parameter sensitivity) of an unknown function is of great importance in stochastic optimization problems. Three kinds of parameter sensitivity estimates using the Monte Carlo method are discussed in this paper. The estimates depend on the number of replications, N, and the change in parameter, Δv. The convergence properties as N→∞ and Δv→0 for these estimates are obtained. The result explains many theoretical and practical issues in the study of discrete event dynamic systems, as well as continuous dynamic systems, by the Monte Carlo method. It is proved that an estimate based on averaging the gradients calculated along each sample path by a perturbation of the path is much better than the other estimates if the output functions are uniformly differentiable with probability one (w.p.1). It is also concluded that in computer simulations one should always choose the same seed for both v and v+Δv in estimating the parameter sensitivity. Combining the results in this paper with existing stochastic approximation algorithms may yield algorithms with faster convergence.