SemChain: Semantic-guided Motion Chains for Skeleton-based Action Recognition

This paper introduces SemChain, a semantic-guided skeleton-based human action recognition framework that integrates structural motion modeling and semantic supervision to address fundamental limitations of existing skeleton-based methods. Current approaches, particularly those based on graph convolutional networks, typically assume that action categories are well defined and separable through joint-level spatio-temporal motion patterns. This assumption is valid in controlled settings but breaks down in realistic scenarios involving fine-grained actions with overlapping or highly similar motion trajectories. This challenge is closely related to kinetic ambiguity, where distinct actions exhibit nearly identical physical dynamics but differ in semantic intent. To overcome kinetic ambiguity, SemChain enhances representation learning from both structural and semantic perspectives within a unified computational intelligence framework. Structurally, a dual-branch architecture combines joint-level modeling with explicit motion-chain modeling to capture coordinated body-part dynamics beyond isolated joints. Semantically, a novel semantic anchor scheme embeds linguistic priors into the visual feature space, enabling adaptive semantic-guided alignment and improving class separability. This design explicitly addresses the limitations of conventional one-hot supervision, which treats action categories as isolated labels and ignores their semantic relationships. Extensive experiments on both NTU RGB+D 60 and BABEL datasets have demonstrated that SemChain achieves strong performance in fine-grained real-world scenarios and competitive results on conventional benchmarks, validating its effectiveness for fine-grained skeleton-based action understanding and its robustness across diverse settings.