Efficient Defense Against Adversarial Attacks on Multimodal Emotion AI Models

Among these applications, emotion AI has emerged as a transformative tool in the educational sector, enabling personalized learning experiences by adapting content and teaching methods based on students’ emotional states. This often involves integrating multimodal data, including text, speech, and facial expressions, to accurately interpret and respond to students’ needs. However, despite its potential, deep learning models are highly vulnerable to adversarial example attacks (AE). These malicious inputs cause incorrect decisions while remaining imperceptible to humans. The risks posed by AEs are particularly concerning in education, where misinterpreting emotional cues may disrupt adaptive learning processes and hinder student engagement. Effectively addressing these challenges is essential. Existing research detects attacks by comparing normal and AEs, but this method struggles with the variability of normal data and the complexity of high-dimensional features, resulting in low detection accuracy and high computational costs. Furthermore, AEs often exhibit unique morphological characteristics, making it difficult to develop a universal detection mechanism. This article proposes a novel method using Poisson distribution to analyze neuron output differences, enhancing model robustness against adversarial attacks and ensuring cross-model generality. By improving adversarial robustness, this method aims to protect emotion AI applications in education, ensuring accurate multimodal data interpretation and supporting adaptive learning systems. Experimental results show that our method performs robustly in multimodal data processing, encompassing text, speech, and facial expressions. Additionally, its strong performance across various attack scenarios highlights the propose method’s generalization capabilities.