AFCN: An Attention-Based Fusion Consistency Network for Facial Emotion Recognition

Due to the local similarities between different facial expressions and the subjective influences of annotators, large-scale facial expression datasets contain significant label noise. Recognition-based noisy labels are a key challenge in the field of deep facial expression recognition (FER). Based on this, this paper proposes a simple and effective attention-based fusion consistency network (AFCN), which suppresses the impact of uncertainty and prevents deep networks from overemphasising local features. Specifically, the AFCN comprises four modules: a sample certainty analysis module, a label correction module, an attention fusion module, and a fusion consistency learning module. Among these, the sample certainty analysis module is designed to calculate the certainty of each input facial expression image; the label correction module re-labels samples with low certainty based on the model’s prediction results; the attention fusion module identifies all possible key regions of facial expressions and fuses them; the fusion consistency learning module constrains the model to maintain consistency between the regions of interest for the actual labels of facial expressions and the fusion of all possible key regions of facial expressions. This guides the model to perceive and learn global facial expression features and prevents it from incorrectly classifying expressions based solely on local features associated with noisy labels. Experiments are conducted on multiple noisy datasets to validate the effectiveness of the proposed method. The experimental results illustrate that the proposed method outperforms current state-of-the-art methods, achieving a 3.03% accuracy improvement on the 30% noisy RAF-DB dataset in particular.