Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems

Abstract A measure of document retrieval system performance called the “expected search length reduction factor” is defined and compared with indicators, such as precision and recall, that have been suggested by other workers. The new measure is based on calculations of the expected number of irrelevant documents in the collection which would have to be searched through before the desired number of relevant documents could be found. Its advantages are: (1) it provides a single index for the property it attempts to measure; (2) it allows for gradations of retrieval status, through the mathematical concept of a “weak ordering”; (3) it evaluates retrieval performance relative to random searching; and (4) it takes into account the amount of relevant material desired by the requester.