On Asymptotic Normality in Stochastic Approximation

A new method, simpler than previous methods due to Chung (1954) and Sacks (1958), is used to prove Theorem 2. 2 below, which implies in a simple way all known results on asymptotic normality in various cases of stochastic approximation. Two examples of application are concerned with Venter's (1967) extension of the RM method and Fabian's (1967) modification of the KW process. Previously, although there was no difficulty in adopting one or the other method, the proofs in various cases had to be done almost ab initio or skipped leaving a gap (see Venter (1967) ). The new proof is similar to that of Chung except that the basic recurrence relation is used to obtain the asymptotic characteristic function rather than limits of all moments. We remark that Lemma 2. 1, a simple corollary to Chung's lemma is used only to obtain condition (2. 2. 4) which is weaker than (2. 2. 3) if = 1 and which corresponds to the usual Lindeberg condition. Both conditions (2. 2. 3) and (2. 2. 4) are weaker than the corresponding condition (3. 4) in Sacks (1958). In what follows (, S, P) will be a probability space, relations between and convergence of random variables, vectors, and matrices will be meant with probability one unless specified otherwise. We shall write Xₙ L if Xₙ is asymptotically L-distributed and Xₙ Yₙ, for two sequences of random vectors, if for any L, Xₙ L if and only if Yₙ L. The indicator function of a set A will be denoted by A, the expectation and conditional expectation by E and EF, respectively. Rᵏ is the k-dimensional Euclidean space the elements of which are considered to be column vectors, R = R¹, R^k k is the space of all real k k matrices. The symbols R, Rᵏ, R^k k, denote sets of all measurable transformations from (, S) to R, Rᵏ R^k k, respectively. The unit matrix in R^k k is denoted by I and | | is the Euclidean norm. With hₙ a sequence of numbers, o (hₙ), O (hₙ), oᵤ (hₙ), Oᵤ (hₙ) denote sequences gₙ, Gₙ, qₙ, Qₙ, say, of elements in one of the sets R, Rᵏ, R^k k such that hₙ^-1 gₙ 0, | hₙ^-1 Gₙ| f for an f R and all n, hₙ^-1 qₙ O uniformly on a set of probability one, | hₙ^-1 Qₙ| K for a K R and all n. In special cases o (hₙ) may be constant on and considered as a sequence with elements in R, Rᵏ or R^k k. Similarly in other cases. For Chung's lemma, which will be frequently referred to, or used later without reference, see Fabian ( (1967), Lemma 4. 2) ; note that it holds with = 0, too.