A deep reinforcement learning-based multimodal framework for credible health information mining on social media

Social media is a very important way to share health information. Still, its use of multiple media types (text, images, etc.) makes it challenging to determine its credibility and how to encourage people to interact. This study proposes a Deep Reinforced Multimodal Health Information Mining (DReMHIM) framework. This transformer-based system uses Robustly Optimized Bidirectional Encoder Representations from Transformers (RoBERTa) for text analysis and Residual Network-50 (ResNet-50) for image feature extraction, with the two modalities combined through dual-attention mechanisms. The model extracts textual and visual features using transformer-based deep learning and employs a dual-attention mechanism to enhance accuracy. Using a Deep Q-Network (DQN) with a Rainbow architecture, DReMHIM refines misinformation detection and content reliability classification. DReMHIM uses a Rainbow Deep Q-Network (Rainbow DQN) to improve credibility classification and sharing techniques. It has been trained on over 3 billion tweets and 31,282 Instagram posts. The results demonstrate that it is 12.5% more effective at identifying false information, 10.7% more effective at finding accurate anecdotal content, 9.4% more effective at classifying items, and 18.6% more effective at maintaining credible content. In comparison, 45.2% less incorrect information is spread. This study effectively addresses the challenge of providing credible health information on social media in real time.