Answers are grounded in this paper and cite their sources.
In order to reduce noise in training data, most natural language crowdsourcing annotation tasks gather redundant labels and aggregate them into an integrated label, which is provided to the classifier. However, aggregation discards potentially useful information from linguistically ambiguous instances.
No takes yet. Share an insight, caveat, or question.
Jamison et al. (2015) studied this question.