The characteristics of facial emotions expressed in Memojis

Memoji represents humans in online communications, and their emotional conveyance is attracting much attention. Despite this, there is a lack of studies exploring the emotional conveyance level of Memoji using various methods. In this study, we examined the emotional quality of Memojis and compared them with photography using the 5-Likert questionnaire and an AI- driven method. An online survey was conducted to collect user assessments of emotional expressions in Memojis. Twenty-eight photographs from the Japanese and Caucasian Facial Expressions of Emotion (JACFEE) database depicting seven emotions were employed. Memojis of these photographs were made with an iPhone using iOS 15.2. The emotional content of individual Memojis was evaluated and compared with those of the photographs. We confirmed each emotion with an AI-driven prediction and analyzed it with human assessment. In general, Memojis were less efficient in communicating emotions than photography. Happiness and sadness showed robust results, while fear and contempt were relatively inaccurately perceived. When partic-ipants were less confident with a Memoji's facial expression, they tended to prefer photography even more. The tendency was coherent between human judgments and AI-supported predictions. We also discuss the limitations and challenges of the proper use of Memojis for improved figure-based and non-verbal communication. • Memojis accurately convey happy, sad, and disgusted faces while failing to express fear, anger, contempt, and surprise. • Memojis failed to convey these four emotions; people tend to confuse fear and surprise, while anger and contempt are mainly perceived as disgust. • Sadness showed high accuracy when Memojis were compared with photographs (Session B), meaning that Memojis achieved power as photography. • Through AI-driven prediction, except for happiness, Memojis convey emotion less reliably than photographs.