Fast supervised dimensionality reduction algorithm with applications to document categorization & retrieval

Retriev al techniques based on dimensionalit y reduction, such as Latent S e m a n tic Indexing (LSI), have been shown to improve the quality of the information being retrieved by c a pturing the latent meaning of the words present in the documents. Unfortunately, the high computational and memory requirements of LSI and its inabilit yto compute an e ective dimensionality reduction in a supervised setting limits its applicability. In this paper we p r e s e n t a fast supervised dimensionality reduction algorithm that is derived from the recen tly dev eloped cluster-based unsupervised dimensionality reduction algorithms. We experimentally evaluate the quality of the low er dimensional spaces both in the context of document categorization and improvements in retrieval performance on a variety of di erent document collections. Our experiments sho w that the lower dimensional spaces computed by our algorithm consistently improve the performance of traditional algorithms such as C4.5, k-nearestneigh bor, and Support V ector Machines (SVM), by a n a verage of 2% to 7%. F urthermore, the supervised lower dimensional space greatly improves the retriev al performance when compared to LSI.