A Biosignal-Driven Wearable System Integrating Surface Electromyography and Kinematics for Interpretable Upper-Limb Assessment

Continuous monitoring of neuromuscular and kinematic behaviors is crucial for effective upper-limb rehabilitation. This study presents a biosignal-driven, wrist-worn multimodal sensing system that combines discrete 12-class functional hand-gesture recognition with an exploratory multimodal physical interpretation of motion deviations. The platform integrates a surface electromyography (sEMG) envelope sensor for macroscopic motor unit recruitment, dual force-sensitive resistors (FSR) for end-effector contact force modulation, and an inertial measurement unit (IMU) for spatial–temporal biokinematics. Data collected from 20 healthy adults (12,000 one-second trials, each containing 100 synchronized time points at 100 Hz) demonstrated that a one-dimensional convolutional neural network (1D-CNN) achieves 81.23% ± 16.74% accuracy under subject-independent leave-one-subject-out (LOSO) cross-validation, capturing natural inter-individual neuromuscular variations. For baseline within-dataset comparison, a class-stratified random split yielded a macro F1-score of 96.59%. To bridge the gap between classification and clinical interpretation, we implemented a multimodal Dynamic Time Warping (DTW) pipeline. This approach successfully identified maximal-difference segments (Hotzones) across the aligned biosignals, enabling a qualitative visualization of candidate multimodal physical deviations (sEMG envelope and IMU/FSR kinematic discrepancies) during gesture execution. By establishing gesture-specific reference distributions from healthy subjects, this cost-effective, low-latency IoT biosensor platform demonstrates preliminary technical feasibility. It lays the groundwork for prospective clinical validation in stroke survivors, aiming to capture compensatory mechanisms and support home-based precision rehabilitation.