Investigating squeeze-and-excitation placement in residual networks for facial emotion recognition

Facial expression recognition (FER) in unconstrained settings remains a core challenge in affective computing. Standard CNNs treat all feature channels uniformly, and existing SE-ResNet variants apply channel attention post-hoc after residual addition, recalibrating a mixed shortcut-residual representation that limits modulation precision. We investigate SE placement within residual networks for FER. ResSENet embeds Squeeze-and-Excitation attention inside each residual branch, recalibrating the learned residual mapping F(X) before identity addition, validated through a five-variant iso-parametric ablation against post-hoc SE-ResNet as a direct control. On FER2013, ResSENet achieves 65.3% test accuracy, outperforming VGGNet-13 by 1.0% with 91% fewer parameters (11.3M vs. 129.0M) and ResNet-18 by 6.5% at a comparable budget. Against its exactly iso-parametric SE-ResNet counterpart (11.3M parameters, 2.50 GFLOPs), ResSENet's 0.6% accuracy gain is accompanied by 22% faster convergence and 36% lower GPU inference latency despite identical compute. ResSENet reaches 260.7 FPS on GPU and 29.1 FPS on CPU, meeting real-time thresholds for edge deployment. Cross-dataset validation on AffectNet and RAF-DB yields 60.3% and 83.6% accuracy, respectively, outperforming post-hoc SE-ResNet by 2.1–2.8%. Targeted SMOTE and conditional GAN augmentation for underrepresented Disgust and confusable Fear classes raise macro F1 from 63.8% to 67.4%. All improvements over baselines are statistically significant (paired t-test, p < 0.05), and GradCAM analysis shows activations aligned with established facial action units.