Key points are not available for this paper at this time.
Dynamic Facial Expression Recognition (DFER) in video sequences presents significant challenges due to the subtle nuances between expressions and inherent dataset imbalances. This study proposes a novel approach leveraging large-scale contrastive visual language pre-training (VLP) models, specifically CLIP and ALIGN, to address these issues. We introduce two key innovations: (1) a concept-guided language-visual feature learning method and (2) a concept-guided sample rebalance technique. The former improves upon traditional text-to-image matching by selecting fine-grained text concepts (e.g., smiling lips, squinting eyes) and constructing corresponding visual concepts using CLIP. This method achieves precise fusion through visual concept backward lookup during training, enhancing semantic information in visual representations. The latter addresses dataset imbalance by synthesizing additional samples for underrepresented expressions and reducing redundancy in overrepresented categories. Our approach is evaluated on three benchmark datasets: DFEW, MAFW, and FERV39k. Results demonstrate significant improvements in both accuracy and robustness of DFER systems compared to existing methods.
Liang et al. (Sat,) studied this question.