Optimizing K-means text document clustering using latent semantic indexing and pillar algorithm

Document clustering is an important tool to help managing the vast amount of digital text document. This paper introduces a new approach to cluster text document. First, text is preprocessed and indexed using inverted index. Then the index is trimmed using TF-DF thresholding. After that, Term Document Matrix is built based on TF-IDF. Next step uses Latent Semantic Indexing to extract important feature from Term Document Matrix. The following process is selecting seeds via Pillar algorithm. Based on determined seeds, K-Means clustering is performed. Experiment result proves that this approach outperforms standard K-Means document clustering.