Answers are grounded in this paper and cite their sources.
Demonstrates improved accuracy in crowdsourcing tasks, indicating enhanced participant engagement.
Crowdsourcing allows many people to complete tasks of various difficulty with minimal recruitment and administration costs. However, the lack of participant accountability may entice people to complete as many tasks as possible without fully engaging in them, jeopardizing the quality of responses. In this paper, we present a dynamic and time efficient solution to the task assignment problem in crowdsourcing platforms. Our proposed approach, CrowdSelect, offers a theoretically proven algorithm to assign workers to tasks in a cost efficient manner, while ensuring high accuracy of the overall task. In contrast to existing works, our approach makes minimal assumptions on the probability of error for workers, and completely removes the assumptions that such probability is known apriori and that it remains consistent over time. Through experiments over real Amazon Mechanical Turk traces and synthetic data, we find that CrowdSelect has a significant gain in term of accuracy compared to state-of-the-art algorithms, and can provide a 17.5\% gain in answers' accuracy compared to previous methods, even when there are over 50\% malicious workers.
No takes yet. Share an insight, caveat, or question.
Qiu et al. (2016) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: