Using interdocument similarity information in document retrieval systems

The first part of this paper reports a comparative study of the document classifications produced by the use of the single linkage, complete linkage, group average, and Ward clustering methods. Studies of cluster member-ship and of the effectiveness of cluster searches support previous findings that suggest that the single linkage classifications are rather different from those produced by the other three methods. These latter methods all pro-duce large numbers of small clusters containing just pairs of documents. This finding motivates the work reported in the second part of the paper, which considers the use of clusters consisting of a document together with that document with which it is most similar. A com-parison of the use of such clusters with conventional best match searches using seven document test collec-tions suggests that the two types of search are of com-parable effectiveness, but they retrieve noticeably differ-ent sets of relevant documents.