Alleviating search uncertainty through concept associations: Automatic indexing, co‐occurrence analysis, and parallel computing

this article, we report research on an algorithmic ap- gather, process, and retrieve information. These systems proach to alleviating search uncertainty in a large infor- provide a wide variety of information and services, rang- mation space. Grounded on object filtering, automatic ing from daily updates of foreign and national news, indexing, and co-occurrence analysis, we performed a movie reviews and clips, law cases, and financial data large-scale experiment using a parallel supercomputer on companies to journal articles, books, trademarks, and ( SGI Power Challenge ) to analyze 400,000/ abstracts in an INSPEC computer engineering collection. Two sys- statistics. However, gaining access to such information is tem-generated thesauri, one based on a combined ob- often difficult. This is due, in large part, to the indetermin- ject filtering and automatic indexing method, and the ism involved in the process by which information is in- other based on automatic indexing only, were compared dexed, and to the latitude searchers have in expressing a with the human-generated INSPEC subject thesaurus. query. Our user evaluation revealed that the system-generated thesauri were better than the INSPEC thesaurus in concept recall, but in concept precision the 3 thesauri were 2. Using Thesauri to Alleviate Search comparable. Our analysis also revealed that the terms suggested by the 3 thesauri were complementary and Uncertainty: Literature Review could be used to significantly increase "variety" in search terms and thereby reduce search uncertainty