New Fast Algorithms for Error Rate-Based Stepwise Variable Selection in Discriminant Analysis

Variable selection is an important technique for reducing the dimensionality in multivariate predictive discriminant analysis and classification. In the past, direct evaluation of the subsets by means of a classifier has been computationally too expensive, rendering necessary the use of heuristic measures of class separation, such as Wilk's Λ or the Mahalanobis distance between class means. We present new fast algorithms for stepwise variable selection based on quadratic and linear classifiers with time complexities which, to within a constant, are the same as those applying measures of class separation. Comparing the new algorithms to previous implementations of classifier-based variable selection, we show that dramatic speed-ups are achieved.