Facial Expression Recognition with Enhanced Relation-Aware Attention and Cross-Feature Fusion transformer

Abstract Face expression recognition(FER) is an important research branch in the field of the computer vision neighborhood. Three prevalent problems in FER tasks that severely impact recognition rates are inter-class similarity, intra-class differences, and facial occlusion issues. Although there have been studies that address some of these issues, none of them can adequately address all three issues in a unified framework. In this paper, we propose a novel dual-branch structure of enhanced relation-aware attention and cross-feature fusion transformer network to comprehensively solve all three issues. Specifically, we design the Enhanced Relation-Aware Attention module to maximize the exploration of more local expression features. At the same time, the Transformer Perceptual Encoder module is adopted to establishing the contextual relationship between individual patches under global information. This greatly alleviates the inter-class similarity problem and the facial occlusion and facial pose transformation problems. On the basis of a dual branch structure, we extract facial image features using facial landmarks features to guide them and design Cross-Feature Fusion Transformer module to deeply cross-fuse two different semantic features. Experiments are performed and results show that our method can greatly alleviated intra-class difference problem with comparison of several traditional methods on three commonly used datasets.