PART-GCN: anatomy-anchored and motion-enhanced adaptive graph topology learning for skeleton-based action recognition

Skeleton-based action recognition is important for intelligent systems that require reliable interpretations of human motion, including rehabilitation monitoring, elder-care assessment, and surveillance. Graph convolutional networks (GCNs) are the dominant approach because they naturally represent the human skeleton as a graph of joints and bones. However, existing adaptive topology-learning methods often optimize graph connectivity with limited structural constraints, allowing the learned topology to deviate from the intrinsic anatomical organization of the human body. In addition, most methods construct adaptive topology primarily from static pose features, limiting their ability to exploit motion dynamics during topology learning. To address these challenges, we propose PART-GCN, a prior-anchored adaptive residual topology learning framework for skeleton-based action recognition. The framework explicitly includes a fixed structural prior while preserving flexibility for action-dependent variations. Specifically, an Anatomy-Anchored Residual Topology (AART) formulation decomposes the adaptive graph into a fixed anatomical prior plus dataset-level static and sample-specific dynamic residuals, making the anatomical reference and learned deviations separately identifiable. A Kinematic Motion Topology Enhancement (KMTE) module further estimates the dynamic residual topology by jointly using pose and velocity information, improving instance-specific inter-joint modeling. We evaluated the proposed framework on three public benchmark datasets: NTU RGB+D, NTU RGB+D 120, and NW-UCLA. PART-GCN achieved the highest mean accuracy on four of the five single-stream benchmarks while remaining highly competitive with recent state-of-the-art methods in the multi-stream setting. On NTU RGB+D 120, PART-GCN achieved mean single-stream accuracies of $$86.7%$$ under the X-Sub protocol and $$88.3%$$ under the X-Set protocol. These results show that topology learning with an explicit anatomical reference and motion-aware dynamic refinement is effective for skeleton-based action recognition.