A truth inference scheme for crowdsourcing using NLP and swin transformers

Crowdsourcing has become a prevalent method for data collection across various domains, offering a scalable and cost-effective solution. However, ensuring the reliability of crowdsourced data remains a significant challenge due to the varying expertise of contributors and the complexity of tasks. Truth inference aims to derive high-quality and accurate answers from heterogeneous and noisy responses for crowdsourcing tasks. In order to address these challenges, we propose a truth inference model that integrates Natural Language Processing with transfer learning using Swin transformers. Unlike traditional transformer architectures, the Swin transformer employs a shifted windowing technique that effectively captures both local and global contextual features in textual data. This approach helps to generate more accurate embedding representations, specifically fine-tuned for nuances of crowdsourced tasks. By incorporating the Swin transformer, our model dynamically refines contributor reliability scores and task difficulty estimates, resulting in a more accurate truth inference. Experimental evaluations on multiple crowdsourcing datasets demonstrate that our approach consistently outperforms state-of-the-art methods in accuracy, scalability, and robustness, particularly under noisy and complex task conditions.