A Monte Carlo Method for Sensitivity Analysis and Parametric Optimization of Nonlinear Stochastic Systems

For high-dimensional or nonlinear problems there are serious limitations on the power of available computational methods for the optimization or parametric optimization of stochastic systems. The paper develops an effective Monte Carlo method for obtaining good estimators of systems sensitivities with respect to system parameters under quite general conditions on the systems and cost functions. The value of the method is borne out by numerical experiments, and the computational requirements are favorable with respect to competing methods when the dimension is high or the nonlinearities “severe.” The method is a type of “derivative of likelihood ratio” method. Jump-diffusion, functional diffusion, and reflected diffusion models of broad types are covered by the basic technique (e.g., the type of limit model that arises in the analysis of queueing systems under heavy traffic, where the boundary reflection conditions are discontinuous). For a wide class of problems, the cost function or dynamics need not be smooth in the state variables; for example, where the cost is the probability of an event or “sign” functions appear in the dynamics. Under appropriate conditions, it is shown that the cost functions are differentiable with respect to the parameters. Since the basic diffusion (or other) model cannot be simulated exactly, two types of readily simulatable approximations are discussed in detail, and estimators of the derivatives of the cost functions for these approximations are obtained and analyzed. It is shown that these estimators and their expectations converge to those for the original problem. Thus, a robustness result for the sensitivity estimators, namely that the derivatives of the cost functions (and their estimators) for the simulatable approximations converge to those for the approximated process is proven. Such results are essential, in any case, if a simulation-based method is to be used with confidence.