Answers are grounded in this paper and cite their sources.
Cyberbullying detection remains a significant challenge in the context of expanding internet and social media usage. This study proposes a novel pretraining methodology for transformer models, integrating Part-of-Speech (POS) information with a unique way of tokenization. The proposed model, based on the ELECTRA architecture, undergoes pretraining and fine-tuning and is referred to as ELECTRA_POS. By leveraging linguistic structures , this approach improves understanding of context and subtle meaning in the text. Through evaluation using the GLUE benchmark and a dedicated cyberbullying detection dataset, ELECTRA_POS consistently delivers enhanced performance compared to conventional transformer models. Key contributions include the introduction of POS-token fusion techniques and their application to improve cyberbullying detection, as well as insights into how linguistic features influence transformer-based models. The result highlights how integrating POS information into the transformer model improves the detection of harmful online behavior while benefiting other natural language processing (NLP) tasks.
No takes yet. Share an insight, caveat, or question.
Azmi et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: