A covariance estimator for small sample size classification problems and its application to feature extraction

A key to successful classification of multivariate data is the defining of an accurate quantitative model of each class. This is especially the case when the dimensionality of the data is high, and the problem is exacerbated when the number of training samples is limited. For the commonly used quadratic maximum-likelihood classifier, the class mean vectors and covariance matrices are required and must be estimated from the available training samples. In high dimensional cases, it has been found that feature extraction methods are especially useful, so as to transform the problem to a lower dimensional space without loss of information, however, here too class statistics estimation error is significant. Finding a suitable regularized covariance estimator is a way to mitigate these estimation error effects. The main purpose of this work is to find an improved regularized covariance estimator of each class with the advantages of Leave-One-Out Covariance Estimator (LOOC) and Bayesian LOOC (BLOOC). Besides, using the proposed covariance estimator to improve the linear feature extraction methods when the multivariate data is singular or nearly so is demonstrated. This work is specifically directed at analysis methods for hyperspectral remote sensing data.