Improved feature representation for robust facial action unit detection

In a Facial Expression Recognition (FER) system, appropriate representation of facial features from relevant face regions play crucial role in robust detection of facial actions units (AUs) under realistic conditions like wide range of illumination variations, presence of tracking errors, inter person expression variations, partial occlusion of faces, etc. In this work, we perform an in-depth analysis of state-of-the-art FER techniques to further understand their performance gaps under realistic conditions. We propose an appropriate Region Of Interest (ROI) selection strategy for each AU and also an appropriately designed robust Local Binary Pattern (LBP) based descriptor that applies spatially spinning bin support for histogram computation. We show that the proposed solutions are capable of addressing performance gaps seen in existing approaches. The ROI strategy present here gives a better trade-off in eliminating inter AU correlations while modeling AUs and minimizes the constraints on the accuracy of facial feature localization. The proposed spin support based feature descriptor provides unique representation for AUs by encoding both appearance and geometry of the facial features in its description and results in a better detection accuracy. We compare the performance of the proposed solutions with the key state-of-the-art techniques and show clear improvement on benchmark databases like CK+, ISL, FACS, JAFFE, MultiPie, MindReading and also on an internally collected real-world data.