End-to-end facial expression detection via dual-stream keypoint-guided architecture

Facial expression detection (FED) holds significant application value in affective computing, yet existing methods predominantly employ cascaded “detection + recognition” schemes that suffer from the absence of end-to-end optimization, poor adaptability of generic detectors, and neglect of facial geometric structures. This paper proposes a dual-stream end-to-end FED framework that simultaneously achieves face localization, expression classification, and 37-point keypoint regression within a unified network through deep coupling of the face branch and keypoint branch. Three cross-branch fusion modules are designed: Keypoint-Guided Multi-Scale Deformable Attention (KG-MSDA) enables high-level features to focus on expression-salient regions, Semantic-Guided Structure Fusion (SGSF) enhances keypoint prediction stability, and Keypoint-to-Face Local Structure Guided Modulation (K2F-LS) leverages global structural configurations to improve bounding box localization. Experiments on the RAF-DB 1 dataset demonstrate that our method achieves 80.65% mAP, a 2.01 percentage point improvement over the baseline, with particularly outstanding performance on expression categories such as Anger and Surprise. Ablation studies validate the effectiveness of each module, while visualization analysis reveals that the network learns expression-related differentiated attention patterns. Our method significantly improves detection accuracy and robustness while maintaining a lightweight design, providing new insights for end-to-end modeling of FED.