Covariance Structure and Convergence Rate of the Gibbs Sampler with Various Scans

SUMMARY This paper presents results on covariance structure and convergence for the Gibbs sampler with both systematic and random scans. It is shown that, under conditions that guarantee the compactness of the Markov forward operator and irreducibility of the corresponding chain, the Gibbs sampling scheme converges geometrically in terms of Pearson χ 2-distance. In particular, for the random scan, the autocovariance can be expressed as variances of iterative conditional expectations. As a consequence, the autocorrelations are all positive and decrease monotonically.