Nonparametric Estimation of Markov Transition Functions

Let \ₙ\\ be a Markov chain having a stationary transition function and assume that the state set is an arbitrary set in a Euclidean space. The state transition law of the chain is given by a function F(y|x) = P\ Xₙ₊₁ \ y|Xₙ = x\, which is assumed defined and continuous for all x. In this paper we give a statistical procedure for determining a function Fₙ(y\ x) on the basis of the sample \ⱼ\\ⁿⱼ₌₁, n = 1, 2,\⋯, and prove that if the chain is irreducible, aperiodic, and possesses a limiting distribution \π, then with probability 1, y|Fₙ(y|x) - F(y|x)| \→ₙ0 for every x such that any open sphere containing x has positive \π probability. This result improves upon a study by Roussas which gives only weak convergence. We demonstrate that a certain clustering algorithm is useful for obtaining efficient versions of our estimates. The potential value of our methods is illustrated by computer studies using simulated data.