Nonparametric Estimation of the Transition Distribution Function of a Markov Process

In 8 the problem of nonparametric estimation in Markov processes has been considered, and estimates of the initial, two-dimensional joint, and transition densities of the process, satisfying a number of optimal properties, have been obtained. In the present paper, under the same nonparametric setup, the attention is centered primarily on the transition distribution function of the process. It will be assumed that the underlying Markov process, defined on a probability space (, a, P) and taking values in the real line R, is (strictly) stationary, and has initial, two-dimensional joint, and transition densities p (), q (, ), and t (x), x R, respectively, relative to the appropriate Lebesgue measures. Let K be a probability density. On the basis of the first n + 1 random variables Xⱼ, j = 1, 2, , n + 1 of the process, we define the random variables pₙ (x), x R, and qₙ (y), y R R (by suppressing the random element) by the following relations equation* 1. 1 pₙ (x) = (nh) ^-1 ⁿ₉=₁ K ( (x - Xⱼ) h^-1) equation* equation* 1. 2 qₙ (y) = qₙ (x, x') = (nh) ^-1 ⁿ₉=₁ K ( (x - Xⱼ) h^-1{2}) K ( (x' - X₉+₁) h^-1{2}), equation* where h = h (n) is a sequence of positive constants satisfying also some additional conditions. We further set equation* 1. 3 tₙ (x' x) = qₙ (x, x') /pₙ (x). equation* Next, by means of pₙ (x) and qₙ (x, x'), define the random variables Fₙ (x) = ˣ- pₙ (z) dz, Gₙ (z x) = ᶻ- tₙ (dx' x). We finally let F () and G (x), x R, be the initial and transition distribution functions of the process. Under suitable conditions on the function K, the sequence \h\, and the process, the main results of this paper are the following. The distribution function Fₙ (x), as an estimate of F (x), obeys the Glivenko-Cantelli theorem. This is Theorem 3. 1. Turning now to the estimate Gₙ (| x) of G (| x), we have been able to establish in Theorem 3. 2 that \|Gₙ (z | x) - G (z | x) a|; z R\ converges to zero, as n, but in the probability sense. This is true for all x R. In Section 4 it is assumed that the rth absolute moment of X₁ exists (for some r = 1, 2, ), and the problem is that of gaining further information about G (x), by estimating its kth moment, to be denoted by m (k; x), for k = 1, 2, , r. By letting the rather simple expression mₙ (k; x) = (nh) ^-1pₙ^-1 (x) ⁿ₉=₁ Xᵏ₉+₁K ( (x - Xⱼ) h^-1) stand for an estimate of m (k; x), it is shown that, as n, mₙ (k; x) m (k; x) in probability, for k = 1, 2, , r and x R. This is the content of Theorem 4. 1. Finally in Section 5 we look into the problem of estimating the quantiles of G (x), and in connection with this, two results are derived. For some p in the interval (0, 1), it is assumed that the pth quantile, (p, x), of G (x) is unique. By defining ₙ (p, x) as the smallest root of the equation Gₙ (z x) = p and using it as an estimate of (p, x), it is proved that, as n, ₙ (p, x) (p, x) in probability (Theorem 5. 1), and (nh) ^1{2} ₙ (p, x) - (p, x) N (0, ² (, x) ) in law. This is Theorem 5. 2, and the variance ² (, x) is explicitly given in that theorem.