PIGMMaLIOn: a Partial Incremental Gaussian Mixture Model with a Low I/O Design

Extracting meaningful information from high data volumes produced on embedded devices relies on machine learning techniques. Traditionally, the generated data are sent to be analyzed on dedicated servers with significant computing resources. However, energy costs and security issues have pushed some machine learning tasks to embedded devices despite their resource limitations. This work focuses on the Gaussian Mixture Model (GMM), a machine learning model used for density estimation and cluster analysis in speaker identification, healthcare, networking, etc. The Expectation-Maximization (EM) algorithm is commonly used to train GMMs. One of the main challenges facing this algorithm when ran on embedded systems is the crippling memory constraints. Indeed, the EM is an iterative algorithm that requires several scans of the dataset. We observed that when the dataset cannot fully reside in the main memory, its execution is dramatically slowed down by I/O movements. In this paper, we present an optimization of the EM algorithm for GMMs that reduces the number of I/O operations thanks to two main contributions: (1) a divide-and-conquer strategy that divides the dataset into chunks, learns the GMM separately on each chunk and combines the results incrementally. By doing so, we prevent data from being swapped several times during the learning process. (2) restricting the training of the GMM on a subset of data whose volume is inferred online using data properties, while producing a good accuracy. On average, our results show a 63% improvement in overall execution time with comparable accuracy.