Stochastic approximation with random truncations, state-dependent noise and discontinuous dynamics

Stochastic approximation algorithms with random truncations, state-dependent noise and discontinuous dynamics are considered in this paper. First, stochastic approximation algorithms with random truncations and additive, state-dependent noise are considered and sufficient conditions for their almost sure convergence are derived. These algorithms are analyzed under assumptions allowing discontinuous dynamics (i.e., the discontinuity of the so-called averaged function) and the state-dependence of the additive noise. The obtained results are then applied to the analysis of stochastic approximation algorithms with random truncations and non-additive noise and sufficient conditions for their almost sure convergence are derived, as well. These algorithms are also analyzed under assumptions which allow discontinuous dynamics {i.e., the discontinuity of both the averaged function and the updating term). Moreover, algorithms with non-additive, state-dependent noise are considered for the case where the noise is a Markov chain controlled by the algorithm state-sequence, while algorithms with non-additive, exogenous noise are analyzed under assumptions allowing the noise to be correlated