Facial Expression Recognition with Limited Labels: A Self-Supervised and Semi-Supervised Approach : Overcoming Label Scarcity in FER

Facial Expression Recognition (FER) faces challenges due to limited labeled data, rendering real-world applications less effective and difficult to implement. This paper proposes a novel three-stage learning framework that integrates self-supervised pre-training, supervised fine-tuning with the newly introduced FaceMix augmentation technique, and semi-supervised learning with an EMA-Teacher model. The self-supervised stage employs a masked autoencoder to learn facial geometry and expression regions from unlabeled images, establishing a robust foundation for expression recognition. FaceMix dynamically adjusts loss weights based on facial region overlap, enhancing the model’s ability to handle diverse expressions while preserving critical facial features. The semi-supervised stage leverages an EMA-Teacher model to generate reliable pseudo-labels for unlabeled data, improving training stability. Experiments on RAF-DB, AffectNet, and FERPlus demonstrate state-of-the-art performance with limited labeled data, achieving 88.23% accuracy on RAF-DB with only 25% labeled data and 64.02% on AffectNet with 5% labeled data. These results significantly outperform existing semi-supervised methods and approach fully supervised performance with substantially fewer labeled samples, making our method highly effective for real-world FER applications where annotation resources are constrained.