Comparison of Initialization Strategies for EM in High-Dimensional Multivariate Diagonal Gaussian Mixture Models

Expectation Maximization (EM) iterations for Gaussian mixture models (GMMs) are highly sensitive to initial parameters, which calls for developing robust initialization methods. For multidimensional GMMs, the problem is even more severe than for univariate GMMs because of the larger number of parameters. For univariate or low-dimensional GMMs, several studies on their initialization have appeared in the literature, whereas research on initializing multivariate, high-dimensional GMMs remains limited. In this study, we compare several initializations for Multivariate Diagonal Gaussian Mixture Models (MDGMMs). In our study, we have included methods already used in the literature for initializing MDGMMs: Hierarchical Clustering, K-means, and random initialization. We have also used a new method, namely, Ensemble Clustering. A review of the existing literature suggests that Ensemble Clustering has not been used previously as an initialization strategy for MDGMM. Several metrics were used to evaluate the clustering quality. Our study demonstrates that Ensemble Clustering, while computationally intensive, is competitive with other methods for initializing MDGMMs.