KeyConcept: Conceptual Search and Pruning Exploiting Concept Relationships

As the number of available Web pages grows, users experience increasing difficulty in finding documents relevant to their interests.One of the underlying reasons for this is that most search engines find matches based on keywords, regardless of their meanings.To provide the user with more useful information, we need a system that better serves user information needs by augmenting queries with information about the user's conceptual framework.This is the goal of KeyConcept, a conceptual search engine.During indexing, KeyConcept classifies documents into concepts selected from a manually-constructed concept hierarchy.During retrieval, KeyConcept ranks documents based on a combination of keyword and conceptual similarity.This paper describes the system architecture and discusses the results of experiments that demonstrate significant improvements in search effectiveness using our approach to conceptual retrieval.Specifically, we compare the precision obtained for a test query set.In addition, when conceptual relationships are used to prune search results, we also achieve significant increases in search result precisions.Finally, when the two approaches, conceptual retrieval and conceptual pruning, are used in combination, we achieve a statistically significant improvement from 36.7% for traditional keyword based search to 63.77% for conceptual search.