A Multi Feature Fusion Search for Facial Expression Recognition Based on Classical Extraction Methods

Abstract The ability to recognize emotional expressions from faces has become an essential component of human-computer interaction (HCI). Recently Oriented FAST and rotated BRIEF (ORB) and Local Binary Patterns (LBP) was used to overcome the limitations of DNN excessive hardware specifications requirements, considering the low hardware specifications used in real-world scenarios. There still exists drawbacks with LBP and ORB, in that LBP is not as resistant to image noise. LBP descriptors are invariant to changing lighting conditions and partial occlusion. Also, when a fixed threshold is utilized under challenging lighting conditions, the ORB algorithm is constrained by its incapacity to extract feature points. We propose a Multi Feature Fusion For Facial Expression Recognition using the algorithms Scale Invariant Feature Transform (SIFT), Histogram Oriented Gradient (HOG), ORB, and LBP. This study proposes a combinatorial blending of least three of these algorithms by looking at the merits of one over the other, also to obtain a novel technique out of the combinatorial schemes, and still obtain better performance of the recognition rates. The proposed method was evaluated on the Extended Cohn Kanade (CK+) and Japanese Famele Facial Expression (JAFFE), and the 2013 Facial Expression Recognition (FER2013) datasets. Based on the merits of our proposed feature extraction schemes, this study explored their respective feature extractions to obtain their individual extracted features from the descriptors. The individual features were then fused together to obtain our multi fused feature, the fused features were then passed onto the classifier for training of our models and image recognitions tasks. This study showed that the proposed algorithm performed well compared to existing state of the art.