Estimating the Infinitesimal Generator of a Continuous Time, Finite State Markov Process

Let (t), t > 0\ be a separable, continuous time Markov Process with stationary transition probabilities Pᵢⱼ(t), i, j = 1, 2, ⋯, M. Under suitable regularity conditions, the matrix of transition probabilities, $P(t)$, can be expressed in the form P(t) = exp tQ, where Q is an M × M matrix and is called the "infinitesimal generator" for the process. In this paper, a density on the space of sample functions over $[0, t)$ is constructed. This density depends upon Q. If Q is unknown, the maximum likelihood estimate Q̂(k, t) = \|q̂ᵢⱼ(k, t)\|, based upon k independent realizations of the process over 0, t) can be derived. If each state has positive probability of being occupied during 0, t) and if the number of independent observations, k, grows larger (t held fixed), then q̂ᵢⱼ is strongly consistent and the joint distribution of the set 1/2(q̂ᵢⱼ - qᵢⱼ) ≠ j (suitably normalized), is asymptotically normal with zero mean and covariance equal to the identity matrix. If k is held fixed (at one, say) and if t grows large, then q̂ᵢⱼ is again strongly consistent and the joint distribution of the set 1/2(q̂ᵢⱼ - qᵢⱼ) ≠ j (suitably normalized), is asymptotically normal with zero mean and covariance equal to the identity matrix, provided that the process (t), t > 0\ is positively regular. The asymptotic variances of the q̂ᵢⱼ are computed in both cases.