Task-aware multiple instance learning for stress detection from facial video data

Stress is a prevalent condition linked to a wide range of mental and physical health disorders. Timely detection of stress is critical for enabling early interventions and promoting long-term well-being. Traditional detection methods based on physiological signals, can be effective, but intrusive or unsuitable for continuous or large-scale deployment. In this study, we propose a video-based stress detection method using top-k Multiple Instance Learning. Our approach is based on the assumption that subjects exhibit a mix of high-intensity and less stress-indicative behaviour during stressful tasks. We employ a temporal feature network with multi-head attention and introduce a conditioning mechanism to account for differences between active (speaking) and passive (non-speaking) tasks. To make better use of limited datasets and weak labels, we incorporate both top-k and bottom-k instances, assuming that bottom-k snippets reflect neutral or less stress-indicative behaviour even in stress-related videos. We validate our approach on our own stress dataset and the publicly available StressID dataset. In a leave-five-subjects-out evaluation, our method achieves high accuracy and F1 scores, outperforming baseline methods while providing interpretable temporal localisation of stress-related behaviour.