Accurate and robust human action recognition (HAR) is critical for enabling intelligent systems to understand and support human activities in dynamic environments. Inclusive physical education requires flexible systems capable of fairly assessing sports actions across individuals with diverse physical abilities. Traditional assessment methods often struggle due to limited labeled data and poor adaptability to varying conditions. In this paper, we propose a Few-Shot Multimodal Sensor Fusion Framework for adaptive sports action recognition, evaluated on the Opportunity and PAMAP2 datasets, to enhance generalizability and robustness. The framework leverages a transformer-based multimodal architecture, combining Vision Transformers (ViT) to extract spatial features from video data with Temporal Convolutional Networks (TCNs) to model temporal patterns from wearable sensors. To support learning from limited labeled samples, we employ few-shot learning techniques, including Contrastive Language-Image Pre-training (CLIP)-based cross-modal alignment, MetaFormer optimization, Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning, and Task-Adaptive Pretraining (TAPT) for domain generalization. Experimental results demonstrate that the framework achieves 94.3% accuracy on Opportunity and 92.8% on PAMAP2, outperforming traditional baselines. High F1-scores, Area Under the Curve of the Receiver Operating Characteristic (AUC-ROC) values above 0.95, and strong few-shot generalization confirm the effectiveness of our approach. These findings highlight the potential of multimodal sensor fusion and few-shot learning for robust, scalable, and inclusive human action recognition, with implications for humanoid robotics, wearable systems, and intelligent physical education environments.
Zhang et al. (Fri,) studied this question.