On-Line Optimization of Simulated Markovian Processes

Let {Z n } be a Markovian process, the transition of which depends on a control parameter x. Let μ x be its invariant law. It is shown that the solution of the optimization problem F(x) := ∫ H(z, x) dμ x (z) = min!, x ∈ S can be found with a recursive estimation procedure of the stochastic approximation-type. The method consists in finding a stochastic quasigradient of F(x) and in adapting the parameter x in the direction of descent. An a.s. convergence is proved and a practical example is given.