Transforming supervised classifiers for feature extraction

Supervised feature extraction is used in data classification and (unlike unsupervised feature extraction) it uses class labels to evaluate the quality of the extracted features. It can be computationally inefficient to perform exhaustive searches to find optimal subsets of features. This article proposes a supervised linear feature extraction algorithm based on the use of multivariate decision trees. The main motivation in proposing this new approach to feature extraction is to reduce the computation time required to induce new classifiers which are required to evaluate every new subset of features. The new feature extraction algorithm proposed uses an approach that is similar to the wrapper model method used in feature selection. In order to evaluate the performance of the proposed algorithm, several tests with real-world data have been performed. The fundamental importance of this new feature extraction method is found in its ability to significantly reduce the computational time required to extract features from large databases.