Prognosis and predictive modeling of hand movements in industrial tasks using StretchSense glove and digital twin simulations

Performance optimization, injury prevention, and skill enhancement in various industrial applications are progressively reliant on predicting and analyzing hand movement patterns, leading to increased efficiency and safety. In this paper, we develop a digital twin framework and investigate the movement patterns to simulate and analyze hand movements across assembly tasks. The study uses a pair of ‘StretchSense Pro Fidelity’ gloves for capturing high-fidelity biomechanical data and tracking movement metrics (finger flexion, extension, and joint angles). We integrate the collected data into our digital twin model to simulate hand interactions in a virtual environment for task evaluation and performance measurement. Our developed model uses a bidirectional Long Short-Term Memory network for prognosis and performance simulation in repetitive or skill-intensive tasks. The deep learning network learns from sequential and temporal data collected from the smart glove. By simulating tasks like precision assembly, tool handling, and control operations, our model identifies optimal hand movement patterns, predicts performance bottlenecks, and offers recommendations for efficiency improvements. The model also evaluates ergonomic factors and predicts muscle fatigue and injury risks by analyzing repetitive motion patterns. The digital twin model can anticipate muscle fatigue based on early indicators and suggest breaks or technique adjustments. It can also tailor training programs to accelerate skill acquisition for novices. We demonstrate how wearable sensor technology in combination with digital twin simulations can optimize and improve industrial processes. Our predictive model delivers real-time feedback, enabling users to make informed decisions that boost efficiency and safety. Preliminary results indicate that this approach can shorten task completion times, improve ergonomic conditions, and reduce repetitive strain injuries. The evaluation shows a high level of metric score in predicting performance metrics, such as task completion time and error rates. Furthermore, the deep learning-based digital twin model successfully identified early signs of muscle fatigue with an accuracy above 85%, and an accuracy of 92% in task detection, enabling more effective task handling.