Discretization of Continuous Markov Chains and Markov Chain Monte Carlo Convergence Assessment

Abstract We show that continuous state-space Markov chains can be rigorously discretized into finite Markov chains. The idea is to subsample the continuous chain at renewal times related to small sets that control the discretization. Once a finite Markov chain is derived from the Markov chain Monte Carlo output, general convergence properties on finite state spaces can be exploited for convergence assessment in several directions. Our choice is based on a divergence criterion derived from Kemeny and Snell, which is first evaluated on parallel chains with a stopping time and then implemented, more efficiently, on two parallel chains only, using Birkhoff's pointwise ergodic theorem for stopping rules. The performance of this criterion is illustrated on three standard examples.