Multilingual language models achieve strong results in emotion classification; however, their robustness under culturally complex and linguistically diverse conditions remains limited. Standard evaluations rely on structured datasets that do not capture phenomena such as code-switching, idiomatic expressions, and indirect affective language, which are common in real-world communication. This study introduces a culturally oriented evaluation framework to analyze the behavior of mBERT, XLM-R, and a fine-tuned variant (XLM-R-FT) under non-standard linguistic conditions. The evaluation combines a multilingual dataset with a subset of manually annotated samples incorporating regional variation in Spanish, Portuguese, and Italian. Results show consistent performance degradation on culturally marked inputs, with reductions of up to 10 F1 points. Fine-tuning improves stability but does not eliminate sensitivity to implicit and idiomatic expressions. Interpretability analysis using LIME and SHAP indicates that predictions are often driven by lexically salient tokens, leading to misinterpretations. The proposed framework enables evaluation beyond standard benchmarks and highlights the need to incorporate cultural and linguistic variability in multilingual emotion analysis.
Villegas-Ch et al. (Tue,) studied this question.