Multi-task learning for smile detection, emotion recognition and gender classification

Facial expression analysis plays a key role in analyzing emotions and human behaviors. Smile detection, emotion recognition and gender classification are special tasks in facial expression analysis with various potential applications. In this paper, we propose an effective architecture of Convolutional Neural Network (CNN) which can jointly learn representations for three tasks: smile detection, emotion recognition and gender classification. In addition, this model can be trained from multiple sources of data with different kinds of task-specific class labels. The extensive experiments show that our model achieves superior accuracy over recent state-of-the-art techniques in all of three tasks on popular benchmarks. We also show that the joint learning helps the tasks with less data considerably benefit from other tasks with richer data.