The retrieval effectiveness of five clustering algorithms as a function of indexing exhaustivity

The retrieval effectiveness of five hierarchical clustering methods (single link, complete link, group average, Ward's method, and weighted average) is examined as a function of indexing exhaustivity with four test collections (CF, Cranfield, Medlars, and Time). Evaluations of retrieval effectiveness, based on three measures of optimal retrieval performance, confirm earlier findings that the performance of a retrieval system based on single-link clustering varies as a function of indexing exhaustively but fail to find similar patterns for other clustering methods. The data also confirm earlier findings regarding the poor performance of single-link clustering in a retrieval environment. The poor performance of single-link clustering appears to derive from that method's tendency to produce a small number of large, ill-defined document clusters. By contrast, the data examined here found the retrieval performance of the other clustering methods to be generally comparable. The data presented here also provide an opportunity to examine the theoretical limits of cluster-based retrieval and to compare these theoretical limits to the effectiveness of operational implementations. Performance standards for the four document collections examined here were found to vary widely, and the effectiveness of operational implementations were found to be in the range defined as “unacceptable.” Further improvements in search strategies and document representations warrant investigation. © 1995 John Wiley & Sons, Inc.