Faster Independent Component Analysis by Preconditioning With Hessian Approximations

Independent Component Analysis (ICA) is a technique for unsupervised exploration of multichannel data that is widely used in observational sciences. In its classic form, ICA relies on modeling the data as linear mixtures of nonGaussian independent sources. The maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data. This problem has been addressed by resorting to quasi-Newton methods, which rely on sparse approximations of the Hessian of the log-likelihood. However, those approximations are not accurate when the ICA model does not hold exactly, as is often the case for real datasets. We propose a new algorithm, dubbed Picard, which makes use of sparse approximate Hessians only as a preconditioner to the L-BFGS algorithm, refining the Hessian approximation from a memory of the past iterates. Extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique, especially on real data.