Asymptotic Properties of Stochastic Approximations with Constant Coefficients

Asymptotic properties (as n → ∞, and then a → 0) of the Stochastic Approximation (SA) algorithm \[ ( * ) ,Xn + 1^a = X_n^a + ah_a ( {X_n^a ,ξ _n^a } ) \] are obtained, where hₐ is not necessarily additive in ξ ₙᵃ. If Ehₐ (x,ξ ₙᵃ ) = g(x) + O(a) and x = g(x) is globally asymptotically stable about a solution xₜ ≡ θ, then the asymptotic properties of \ (Xₙᵃ - θ ) / √ a \ ≡ \ Uₙᵃ \ are developed. In particular, it is shown that (as a → 0) a natural continuous parameter interpolation of the tail part of \ Uₙᵃ \ converges weakly to a stationary Gauss-Markov process, from which the asymptotic properties of \ Uₙᵃ \ and \ Xₙᵃ \ can be obtained for small a. The conditions on \ ξ ₙᵃ \ are reasonable from the point of view of the usual applications to adaptive systems and identification. These results seem to be the first of their type for SA’s with constant coefficients. Some rate of convergence results for classical SA’s are improved. Also, an application of (*) to a problem of tracking the time varying parameters of a linear system is discussed, and a limit theorem obtained. Because in the usual practical implementations of SA to problems in systems theory, the gain sequence \ aₙ \ does not normally go to zero (due to considerations of robustness and nonstationarities), these results are of particular importance.