Fusion of static and temporal predictors for unconstrained facial expression recognition

Facial expression research on unconstrained spontaneous expressions has benefited from recent advances in feature extraction, dimensionality reduction, and classification techniques. While the facial action coding relies on temporal predictors, state-of-the-art facial expression recognition techniques have been slow to adapt to temporal methods. Further, despite strong evidence of sparsity in the visual cortex, few approaches to facial understanding utilize sparse representations. This paper proposes using a temporal detector based upon facial dynamics, a static expression detector based on sparse representations, and an advanced temporal-sparse fused emotion estimator. Our approach leverages techniques from both computer vision and human brain research to produce a state-of-the-art emotion estimator on unconstrained faces in natural conditions.