Human-in-the-Loop AI for Analysis of Free Response Facial Expression Label Sets

Facial expressions (FEs) communicate a rich variety of social, grammatical, and affective signals. However, the most generally accepted set of recognizable FEs remains limited to seven basic displays of emotion: happiness, sadness, fear, anger, disgust, surprise and contempt. To develop intelligent virtual agents capable of interpreting and synthesizing nuanced facial behavior, we need a more complete lexicon. One roadblock has been the limiting nature of forced-choice study designs, the most common paradigm for investigating observer judgements of FEs. However, there has been no consensus on an objective way to evaluate alternative free response designs.