Arousal and Valence Estimation for Visual Non-Intrusive Stress Monitoring

As the capabilities and usefulness of advanced Unmanned Aerial Vehicles (UAVs) increase, communication between the operator and these intelligent systems is becoming a very important factor for mission success. In this context, automatic stress detection is becoming a key research topic in emotion analysis. Stress can be estimated by means of an array of intrusive sensors or via the measurement of some biological markers (e.g. cortisol levels). However, these approaches are not appropriate in many cases of human-machine interactions. In this paper, we propose a deep learning-based psychological stress level estimation approach. The goal is to identify the region where the emotional state of the operator projects in the space defined by the latent dimensional emotions of arousal and valence. The stress region is well defined in this space according to prior works in psychology. The proposed predictive model first extracts and aligns the operator's face, then generates embeddings from a pre-trained face model. These embeddings are then used to train two different architectures, a hierarchical temporal CNN and a LSTM with an Attention Weighted Average layer. Since we deal with naturalistic behavior in a context of operator-machine interaction, the One-Minute Gradual-Emotion Behavior Challenge (OMG) dataset is used for the validation of continuously estimated arousal/valence levels.