Group-by and top-k are fundamental constructs in database queries. However, the criteria used for grouping and ordering certain types of data -- such as unlabeled photos clustered by the same person ordered by age -- are difficult to evaluate by machines. In contrast, these tasks are easy for humans to evaluate and are therefore natural candidates for being crowd-sourced.
No takes yet. Share an insight, caveat, or question.
Davidson et al. (2013) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: