Cyclic minimizers, majorization techniques, and the expectation-maximization algorithm: a refresher

Many parameter estimation problems in signal processing can be reduced to the task of minimizing a function of the unknown parameters. This task is difficult owing to the existence of possibly local minima and the sharpness of the global minimum. In this article we review three approaches that can be used to minimize functions of the type encountered in parameter estimation problems. The first two approaches, the cyclic minimization and the majorization technique, are quite general, whereas the third one, the expectation-maximization (EM) algorithm, is tied to the use of the maximum likelihood (ML) method for parameter estimation. The article provides a quick refresher of the aforementioned approaches for a wide readership.