Computational and Inferential Difficulties with Mixture Posterior Distributions

Abstract This article deals with both exploration and interpretation problems related to posterior distributions for mixture models. The specification of mixture posterior distributions means that the presence of k! modes is known immediately. Standard Markov chain Monte Carlo (MCMC) techniques usually have difficulties with well-separated modes such as occur here; the MCMC sampler stays within a neighborhood of a local mode and fails to visit other equally important modes. We show that exploration of these modes can be imposed using tempered transitions. However, if the prior distribution does not distinguish between the different components, then the posterior mixture distribution is symmetric and standard estimators such as posterior means cannot be used. We propose alternatives for Bayesian inference for permutation invariant posteriors, including a clustering device and alternative appropriate loss functions.