Pruning Noisy Bases in Discriminant Analysis

The success of linear discriminant analysis (LDA) is due in part to the simplicity of its formulation, which reduces to a simultaneous diagonalization of two symmetric matrices A and B;. However, a fundamental drawback of this approach is that it cannot be efficiently applied wherever the matrix A is singular or when some of the smallest variances in A are due to noise. In this paper, we present a factorization of A(-1) and a correlation-based criterion that can be readily employed to solve these problems. We provide detailed derivations for the linear and nonlinear classification problems. The usefulness of the proposed approach is demonstrated thoroughly using a large variety of databases.