Consistent Estimation of a Mixing Distribution

A maximum-penalized-likelihood method is proposed for estimating a mixing distribution and it is shown that this method produces a consistent estimator, in the sense of weak convergence. In particular, a new proof of the consistency of maximum-likelihood estimators is given. The estimated number of components is shown to be at least as large as the true number, for large samples. Also, the large-sample limits of estimators which are constrained to have a fixed finite number of components are identified as distributions minimizing Kullback-Leibler divergence from the true mixing distribution. Estimation of a Poisson mixture distribution is illustrated using the distribution of traffic accidents presented by Simar.