The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence

A generalisation of the ecm algorithm (Meng & Rubin, 1993), which is itself an extension of the em algorithm (Dempster, Laird & Rubin, 1977), can be obtained by replacing some CM-steps of ECM, which maximise the constrained expected complete-data loglikelihood function, with steps that maximise the correspondingly constrained actual likelihood function. This algorithm, which we call ecme algorithm, for Expectation/Conditional Maximisation Either, shares with both em and ecm their stable monotone convergence and basic simplicity of implementation relative to competing faster converging methods. Moreover, ecme can have a substantially faster convergence rate than either EM or ECM, measured using either the number of iterations or actual computer time. There are two reasons for this improvement. First, in some of ECME'S maximisation steps, the actual likelihood is being conditionally maximised, rather than a current approximation to it, as with EM and ECM. Secondly, ecme allows faster converging numerical methods to be used on only those constrained maximisations where they are most efficacious. Illustrative ecme algorithms are presented with both closed-form and iterative CM-steps, which demonstrate the faster rate of convergence and the associated easier assessment of convergence. Also, theoretical expressions are presented and illustrated regarding the rate of convergence of ecme. Finally, relationships with Markov chain Monte Carlo methods are discussed.