Multi-Task Learning for Sports Training Monitoring via Multimodal Fusion of IMU and Human-Body Capacitance Signals

For training monitoring in free-living fitness scenarios, this paper addresses the limitations of inertial measurement units (IMUs) in cross-body information perception, accumulation of integration drift, and limited performance in recognizing low-amplitude movements. We introduce human-body capacitance signals (HBC) as a complementary modality and propose a channel-level graph attention–temporal convolution multimodal multi-task learning framework. Unlike approaches relying solely on local inertial measurements, HBC signals capture the overall movement correlation between distal body parts (such as the lower limbs or weighted equipment) and the torso or sensor-worn locations through body electrostatic coupling. This allows explicit modeling of cross-limb dependencies, compensating for the IMU’s limitations in characterizing full-body coordinated movements under local sensing conditions. The proposed method models each IMU and HBC channel as graph nodes, combining physical priors and data-driven edge sets. A graph attention network (GAT) adaptively learns cross-channel and cross-modality dependency structures, while a temporal convolutional network extracts multi-scale rhythmic features. The unified framework jointly optimizes three tasks: activity recognition, repetition counting, and training intensity estimation. Cross-subject (LOSO) evaluation results on the RecGym dataset show that the model achieves an average recognition accuracy of 0.894 (Kappa 0.871) across 10 subjects. In subsets involving low-amplitude movements (such as light dumbbell exercises and small-range joint adjustments), the fusion model demonstrates more stable recognition performance, maintaining an accuracy around 0.9, outperforming the IMU-only model. Additionally, the model achieves MAE values of 0.234 for counting and 0.156 for intensity estimation. Further modality comparisons and attention analysis reveal that the fusion of IMU and HBC not only enhances recognition robustness in low-dynamic scenarios but also provides a physically interpretable distribution of channel importance, offering an efficient and interpretable multimodal solution for wearable sports training monitoring.