As the scale of data grows for machine learning, annotating data accurately is extremely time-consuming and with high economic costs. To alleviate this dilemma, crowdsourcing has been widely used for data collection and annotation. Learning from crowds gradually becomes a popular machine learning paradigm, where learning models are trained with datasets labeled by different crowd workers. Therefore, it inevitably introduces label noise, which seriously degrades the performance of models. Previous research reveals that modeling the process of generating crowdsourced labels is a powerful tool to solve label noise. However, the sparsity of crowdsourced labels in practice hinders the effectiveness of modeling the capabilities of workers. To address this issue, this paper proposes a novel Multi-Feature Dynamic Fusion Annotation Generation Network (MFDFAGen-Net). The proposed model can dynamically rectify the learning process by using the correlation between noise transition and instance dependency, which enables the learned worker-capability and instance-rectification confusion matrices to optimize with each other. In addition, two confusion matrices are fused with both worker and instance features to enhance the expression of the capability of workers. Theoretical analysis indicates that MFDFAGen-Net can better learn the reliability and bias of instances and workers, providing more fine-grained knowledge. Extensive experimental results on a variety of synthetic and real-world datasets consistently also show that the proposed MFDFAGen-Net significantly outperforms many state-of-the-art methods.
Zhang et al. (2026) studied this question.