Convolutional Neural Networks (CNNs) have made significant progress in video-based human action recognition (HAR). However, single-modality CNN approaches often fail to jointly capture the multi-scale temporal dynamics of actions and to focus on the most informative body regions within each clip. In this paper, we propose the Multimodal SlowFast architecture (MM-SF) that combines the complementary strengths of RGB video and estimated 2-D skeleton (pose) data while enabling continuous feature-level exchange between the two modalities. The architecture comprises two RGB pathways operating at complementary temporal rates—a Slow pathway for static semantics and a Fast pathway for fine-grained motion—together with a dedicated Pose pathway that captures the temporal dynamics of human body kinematics. Bidirectional lateral connections between the RGB and Pose pipelines allow stage-wise cross-modal feature interaction, going beyond the score-level fusion used in prior work. Moreover, every pathway backbone is augmented with a Convolutional Block Attention Module (CBAM), yielding an Attention-ResNet backbone that sharpens spatial and channel representations. The MM-SF captures spatio-temporal patterns within each sampled clip at multiple temporal rates (short-to-mid-range context within the SlowFast sampling window, rather than full-video global reasoning). Experimental results show that the proposed framework achieves competitive performance on UCF-101 and state-of-the-art results on the challenging Diving-48 dataset.
Lasri et al. (Mon,) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: