Exploring Coherent Motion Patterns via Structured Trajectory Learning for Crowd Mood Modeling

Crowd behavior analysis has recently attracted extensive attention in research. However, the existing research mainly focuses on investigating motion patterns in crowds, while the emotional aspects of crowd behaviors are left unexplored. Analyzing the emotion of crowd behaviors is indeed extremely important, as it uncovers the social moods that are beneficial for video surveillance. In this paper, we propose a novel crowd representation termed crowd mood. Crowd mood is established based upon the discovery that the social emotional hypothesis of crowd behaviors can be revealed by investigating the spacing interactions and the structural levels of motion patterns in crowds. To this end, we first learn the structured trajectories of crowds by particle advection using low-rank approximation with group sparsity constraint, which implicitly characterizes the coherent motion patterns. Second, rich emotional motion features are explicitly extracted and fused by support vector regression to reflect social characteristics. In particular, we construct weighted features in a boosted manner by considering the features' significance. Finally, crowd mood is intuitively presented as affective curves to track the emotion states of the crowd dynamics, which is robust to noise, sensitive to semantic shift, and compact for pattern expressions. Extensive evaluations on crowd video data sets demonstrate that our approach effectively models crowd mood and achieves significantly better results with comparisons to several alternative and state-of-the-art approaches for various tasks, i.e., crowd mood classification, global abnormal mood detection, and crowd emotion matching.