Approximate Posterior Distributions for Incomplete Data Problems

Summary We consider the problem of developing a simple approximation to a posterior distribution arising from incomplete data sampling. We compare approximations based on the normal distribution and upon conjugate distributions, theoretically where feasible and in some numerical examples. Two general conclusions can be made on the basis of our numerical work. First, when there is missing data, approximations which fail to take loss of information into account give overly concentrated posterior distributions. Second, the Gaussian approximation matching mode and observed Fisher information is quite good with large sample sizes and true posteriors which are not highly skewed. Further work delimiting these conditions more precisely will be useful. Finally, the use of a conjugate posterior which matches both mode and information performs well in all of our examples, and is decidedly superior to the Gaussian approximation with small samples.