Answers are grounded in this paper and cite their sources.
Relevance evaluation is an essential part of the development and maintenance of information retrieval systems. Yet traditional evaluation approaches have several limitations; in particular, conducting new editorial evaluations of a search system can be very expensive. We describe a new approach to evaluation called TERC, based on the crowdsourcing paradigm, in which many online users, drawn from a large community, each performs a small evaluation task.
No takes yet. Share an insight, caveat, or question.
Alonso et al. (2008) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: