Decode the Dark Side of the Language: Applications of LLMs in the Dark Web

Previous studies have discovered that Transformers pre-trained on large scale datasets have a great advantage over traditional methods and can achieve human-level performance in many NLP tasks. Various encoder-only models like BERT and RoBERTa are used in the researches of the language in the Dark Web and most of them have achieved state-of-the-art performance. However, decoder-only models are not frequently used as they were considered to be unsuitable for NLU tasks until recent advancements in large language models. In this paper, we discuss the application of large scale decoder-only models in NLP tasks of real-world Dark Web research. We explore the capabilities of these models with the most realistic datasets in this specific domain. Surprisingly, results show that decoder-only models can achieve similar or better performance compared with state-of-the-art encoder-only model in some cases and show their own unique advantages in such field. We also discuss the limitations of these models and the challenges associated with their application in such field. Our study unveils the potential of decoder-only models in NLP tasks of real-world Dark Web research.