A Sandwich Proof of the Shannon-McMillan-Breiman Theorem

Let ₜ\ be a stationary ergodic process with distribution P admitting densities p(x₀,…, xₙ₋₁) relative to a reference measure M that is finite order Markov with stationary transition kernel. Let IM(P) denote the relative entropy rate. Then n⁻¹log p(X₀,…, Xₙ₋₁) → IM(P) a.s. (P). We present an elementary proof of the Shannon-McMillan-Breiman theorem and the preceding generalization, obviating the need to verify integrability conditions and also covering the case IM(P) = ∞. A sandwich argument reduces the proof to direct applications of the ergodic theorem.