A reliability-oriented multimodal ensemble framework for fake news detection in armed conflict crisis situations

Social media platforms where people can post fake news threaten public trust, social stability, and the informed nature of people’s choices because of the rapid growth of fake news. Multimedia content is widely accessible today and has added to the complication of misinformation through the use of multimedia (including images, audio, video) in combination with textual information, which often contain emotional language to create narratives that deceive consumers of information. There has been much research over the last few years that has explored ways to detect fake news on multiple platforms, and while this research has resulted in the development of many multimodal fake news detection systems, the majority of these current methods rely on single-classifier architectures and fairly simple fusion strategies that do not provide much in terms of robustness, generalization, or reliability in terms of detection under noisy and ambiguous conditions. To address these limitations, this study presents a novel multimodal fake news detection framework that incorporates both textual and visual representation learning based on transformer models, affective feature modeling, and a stacked ensemble decision-fusion approach. The textual representation learning module in the proposed framework makes use of the XLM-R model, whereas the visual representation learning module uses the ViT-B/16 model. Finally, the affective modeling module utilizes pretrained sentiment and emotion models for capturing and distinguishing the affective signals. The predictions made by the multimodal representations are combined through the meta-classifier to generate the final classification result. In this regard, the proposed architecture enables robust cross-modal reasoning and probabilistic calibration. In what follows, we discuss the empirical results of extensive experiments conducted using four large-scale benchmark datasets (Fakeddit, Ukraine-GitHub, Weibo, and MediaEval), which show that the proposed multimodal detection architecture always outperforms the state-of-the-art multimodal detection approaches proposed recently. The proposed framework performs competitively on four benchmark datasets and achieves the accuracy of 94.66%, 98.23%, 98.12%, and 97.98% on Fakeddit, Ukraine-GitHub, Weibo, and MediaEval, respectively. Evaluations and ablation studies confirm the efficacy of the proposed multimodal ensemble approach under the given experimental settings.