Facial Expression Recognition from 3D Facial Landmarks Reconstructed from Images

Direct classification of normalized and flattened 3D facial landmarks reconstructed from 2D images is proposed in this paper for recognizing eight types of facial expressions depicting the emotions of- sadness, anger, contempt, disgust, fear, happiness, neutral and surprised. The first stage is the 3D projection of 2D facial landmarks. The pre-trained convolutional Face Alignment Network (FAN) proposed recently for 2D/3D face alignment is used for the purpose. The 3D cartesian coordinates are translated to the spherical coordinate system prior to the classification stage. A variety of classifiers are tried and tested for classifying the 68 facial landmarks, for both the coordinate systems. The benchmark CK+ video dataset is used for the experimentation; the last frame of each video that depicts the peak of each emotion is used as the input image. The experimental results indicate that direct classification of normalized and flattened 3D facial landmarks in the spherical coordinate system yields the highest accuracy for the support vector machine classifier with grid search for determining optimal parameters.