Approximating Priors by Mixtures of Natural Conjugate Priors

SUMMARY The purpose of this paper is twofold: First, to show that a natural enlargement of the class of natural conjugate priors–namely, mixtures of natural conjugate priors–also leads to mathematically tractable solutions. Secondly, to show that this enlargement is “adequate” in that any prior may be arbitrarily closely approximated by a suitable member of this class. Specifically, a general method for approximating an arbitrary prior density is first given with approximation (convergence) defined pointwise or in the L 1 or total variation senses. Corresponding posterior densities are shown to provide approximations in corresponding senses, whether or not the approximating priors are those given herein. Then formulas are given for posterior densities when the prior is a mixture of natural conjugate priors, and it is shown that an arbitrary prior density and its posterior density may be approximated by such a mixture and its posterior. In these senses, Bayesians may find it satisfactory to confine attention to mixtures of natural conjugate priors. The results are illustrated by several exponential family examples.