Asymptotic Almost Sure Efficiency of Averaged Stochastic Algorithms

First, we define the notion of almost sure efficiency for a decreasing stepsize stochastic algorithm, and then we show that the averaging method, which gives asymptotically efficient algorithms, also gives asymptotically almost surely efficient algorithms. Moreover, we prove that the averaged algorithm also satisfies a lawof the iterated logarithm, as well as an almost sure central limit theorem.