Multimodal AI approach to platform hate

A multimodal AI approach for detecting hate speech in social media and online platforms is explored in this study. It integrates AI techniques (Natural Language Processing (NLP), image processing, video processing and moderation models) to analyse multimodal communications across media platforms. The approach is illustrated using an interactive digital platform developed for analysing communications during the COVID-19 pandemic. The platform supports (a) collecting multimedia content from online sources; (b) processing text, images and videos; and (c) visualising results through an interactive dashboard. The findings indicate that publicly available AI tools for hate speech detection, in this case OpenAI moderation models (text-moderation 2023 and omni-moderation 2026) applied to text, OCR-extracted image text and transcribed video audio, are unreliable for identifying hate speech in real-life data. It is proposed that multimodal computational approaches informed by social semiotic theory would improve methods for automated hate speech detection.