Engagement Modeling in Dyadic Interaction

In the recent years, engagement modeling has gained increasing attention due the important role it plays in human-agent interaction. The agent should be able to detect, in real time, the engagement level of the user in order to react accordingly. In this context, our goal is to develop a computational model to predict engagement level of the user in real time. Relying on previous findings, we use facial expressions, head movements and gaze direction as predictive features. Moreover, engagement is not only measured from single cues, but from the combination of several cues that arise over a certain time window. Thus, for better engagement prediction, we consider the variation of multimodal behaviors over time. To this end, we rely on LSTM that can jointly model the temporality and the sequentiality of multimodal behaviors.