The Second Skin: A Wearable Sensor Suite That Enables Real-Time Human Biomechanics Tracking Through Deep Learning

This is the authors' abstract. We don't add key points for this paper.

OBJECTIVE: Real-time determination of human kinematics and kinetics could advance biomechanics research and enable valuable applications of biofeedback and generalizable exoskeleton control. This work aims to investigate a task-independent, user-independent method for obtaining precise real-time joint state estimation across lower-body joints during a wide variety of tasks. METHODS: We developed a generalizable sensing approach using a suit comprised of inertial measurement units (IMUs) and pressure insoles. With the suit, we collected a dataset of 33 tasks commonly performed during construction and hazardous waste cleanup (N = 10). We then trained deep learning user-independent, task-agnostic models to estimate joint lower-body kinematics and dynamics using only worn sensor data. We likewise computed joint kinematics and dynamics analytically from sensor data to serve as a comparison tool for model results. RESULTS: Our models achieved overall angle estimation root-mean-squared-errors (RMSE) of 6.56±.92°, 8.60±1.01°, 7.58±.89°, and 6.00±.73° compared to 13.9±.1.3°, 15.31±1.0°, 10.76±.70°, and 7.56±.48° via analytical methods at the lower back, hip, knee, and ankle, respectively. Likewise, our models achieved overall normalized moment estimation RMSEs of .207±.069 Nm/kg, .242±.044 Nm/kg, .202±.038 Nm/kg, and .193±.034 Nm/kg compared to .306±.036 Nm/kg, .407±.021 Nm/kg, 1.18 ±.022 Nm/kg, and 1.73±.071 Nm/kg via analytical methods at the lower back, hip, knee, and ankle, respectively. CONCLUSION: These results are comparable to other state-of-the-art wearable sensing systems, establishing deep learning as a viable sensing approach that generalizes to new users and tasks. SIGNIFICANCE: This work shows promise for enabling accurate real-world biomechanical data collection and enhancement of biofeedback systems and wearable robot control.