The Strong Ergodic Theorem for Densities: Generalized Shannon-McMillan-Breiman Theorem

Let ₁, X₂,⋯\ be a stationary process with probability densities f(X₁, X₂,⋯, Xₙ) with respect to Lebesgue measure or with respect to a Markov measure with a stationary transition measure. It is shown that the sequence of relative entropy densities (1/n)log f(X₁, X₂,⋯, Xₙ) converges almost surely. This long-conjectured result extends the L¹ convergence obtained by Moy, Perez, and Kieffer and generalizes the Shannon-McMillan-Breiman theorem to nondiscrete processes. The heart of the proof is a new martingale inequality which shows that logarithms of densities are L¹ dominated.