Machine learning for human emotion recognition: a comprehensive review

Abstract Emotion is an interdisciplinary research field investigated by many research areas such as psychology, philosophy, computing, and others. Emotions influence how we make decisions, plan, reason, and deal with various aspects. Automated human emotion recognition (AHER) is a critical research topic in Computer Science. It can be applied in many applications such as marketing, human–robot interaction, electronic games, E-learning, and many more. It is essential for any application requiring to know the emotional state of the person and act accordingly. The automated methods for recognizing emotions use many modalities such as facial expressions, written text, speech, and various biosignals such as the electroencephalograph, blood volume pulse, electrocardiogram, and others to recognize emotions. The signals can be used individually(uni-modal) or as a combination of more than one modality (multi-modal). Most of the work presented is in laboratory experiments and personalized models. Recent research is concerned about in the wild experiments and creating generic models. This study presents a comprehensive review and an evaluation of the state-of-the-art methods for AHER employing machine learning from a computer science perspective and directions for future research work.