What question did this study set out to answer?

To introduce and evaluate novel cumulated gain-based metrics that reward information retrieval algorithms for ranking highly relevant documents ahead of marginally relevant ones.

October 1, 2002ACM Transactions on Information Systems

Cumulated gain-based evaluation of IR techniques

View Full Paper

Ask AI

Bookmark

Authors

KJKalervo JärvelinTampere University JKJaana KekäläinenTampere University

Discussion

Member takes

Implication

Methodological evaluation demonstrates graded cumulated gain metrics in benchmark retrieval datasets, indicating improved sensitivity to highly relevant ranked documents.

Key Points

To introduce and evaluate novel cumulated gain-based metrics that reward information retrieval algorithms for ranking highly relevant documents ahead of marginally relevant ones.
Designed three novel evaluation measures: cumulative gain across rank positions, discounted cumulative gain to penalize late-retrieved documents, and normalized gain relative to ideal rankings.
Tested the metrics using sample system runs across 20 queries from the TREC-7 benchmark evaluated against a four-point graded relevance scale.
Cumulated gain measures successfully credited retrieval systems for identifying and prioritizing highly relevant documents near the top of the ranked list.
The metrics enabled valid statistical significance testing of effectiveness differences between competing retrieval techniques and facilitated graphical analysis from a user perspective.

Cite This Study

Järvelin et al. (2002) studied this question.

synapsesocial.com/papers/69d8ab72a5ecc596b5d182a2 https://doi.org/10.1145/582415.582418

View Full Paper

Ask AI

Bookmark

Implication

Methodological evaluation demonstrates graded cumulated gain metrics in benchmark retrieval datasets, indicating improved sensitivity to highly relevant ranked documents.

Cumulated gain-based evaluation of IR techniques

Discussion

Member takes

Implication

Key Points

Cite This Study

Cumulated gain-based evaluation of IR techniques

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider