Benchmarking a Large Twitter Dataset for Arabic Emotion Analysis

Abstract Emotion analysis of social media content, e.g., Twitter, is a useful tool forunderstanding people’s reactions during critical events such as Covid-19pandemic. Arabic annotated emotion datasets are nonetheless scarce, whichconsequently affects the accuracy of emotion detection applications, and limittheir usefulness. During the first year of the pandemic, a large Arabic EgyptianCOVID-19 Twitter Dataset (ArECTD) was collected and annotated employing amix of manual annotation and semi-supervised self-learning technique. ArECTDis compromised of approximately 78K tweets and ten emotion labels making itone of the largest available Arabic emotion datasets. The mixed annotationapproach is particularly promising for handling the demanding annotation taskand potentially growing the number of Arabic emotion datasets. The paper aimsto examine the quality of the annotation of ArECTD by measuring the lexicalcorrelates of each emotion. Moreover, content-based analysis was conducted as apractical case study to identify correlations between the detected emotions andthe Egyptian government’s decisions during the pandemic. Furthermore,classification models are developed for ArECTD by fine-tuning thestate-of-the-art Arabic deep learning transformer models, AraBERT andMARBERT, achieving accuracies of 70.01% of 72.5%, respectively. Thegeneralization of the best ArECTD classification model to two other datasetsfrom different domains was studied using transfer learning.