BMHA-Net: A two-branch multiscale hierarchical attention network for depression severity estimation from facial videos

Depression is a prevalent mental disorder that imposes a substantial public-health and socioeconomic burden, while conventional assessments based on rating scales and clinical interviews are constrained by subjectivity and limited professional resources, hindering large-scale, low-cost, and consistent assessment. Facial-video-based automated modeling provides a non-invasive alternative for depression severity estimation and has attracted increasing attention because facial dynamics may provide informative behavioral cues associated with depression. However, depression-related visual cues are often subtle, sparsely distributed, and sensitive to head pose, illumination, and inter-subject variability, while existing multi-branch attention designs may introduce additional parameter and computational overhead, making the balance between prediction accuracy and model complexity an important issue. To address these challenges, we propose BMHA-Net, a two-branch multiscale hierarchical attention network that enhances fine-grained facial cues through high-resolution local and shifted-window self-attention, captures global spatial dependencies through low-resolution global attention, and strengthens appearance–geometry complementarity via bidirectional cross-branch interaction and adaptive multi-scale fusion. Experiments on AVEC2014 demonstrate that BMHA-Net reduces MAE from 8.86 to 6.30 (28.9% relative) and RMSE from 10.86 to 8.47 (22.0% relative) compared with the official baseline. Under the same visual-only input setting, BMHA-Net achieves lower FLOPs than several recent vision-based methods with reported computational costs while maintaining competitive regression performance. Additional experiments on DAIC-WOZ further demonstrate its adaptability to pre-extracted visual descriptors under a different depression-score protocol.