Clip-GCN: an adaptive detection model for multimodal emergent fake news domains

Abstract Emergent news is characterized by few labels, and news detection methods that rely on a large number of labels are difficult to apply to learned features for emerging events and are ineffective in coping with less labeled emergent news detection. To address the challenge of limited labeled data, this study first establishes a scenario for detecting breaking news, ensuring that the domain of detecting events is distinct from the domain of historical events. Secondly, we propose the Clip-GCN multimodal fake news detection model. The model utilizes the Clip pre-training model to perform joint semantic feature extraction of image-text information, with text information as the supervisory signal, which solves the problem of semantic interaction between modalities. Meanwhile, considering the domain attributes of news, the model is trained to extract inter-domain invariant features through Adversarial Neural Network ideation, and intra-domain knowledge information is utilized through graph convolutional networks (GCN) to detect emergent news. Through an extensive number of experiments on Chinese and English datasets from two major social media platforms, Weibo and Twitter, it is demonstrated that the model proposed in this paper can accurately screen multimodal emergent news on social media with an average accuracy of 88.7%. The contribution of this study lies not only in the improvement of model performance but also in the proposal of a solution for the challenges posed by limited labels and multimodal breaking news. This provides robust support for research in related fields.