What question did this study set out to answer?

The aim is to develop a deep learning-based controller for a robotic hand to mimic human movements without complex physical models.

January 23, 2026Open Access

From Video to Torque: A Deep Learning approach to Robotic Hand Control

Key Points

The aim is to develop a deep learning-based controller for a robotic hand to mimic human movements without complex physical models.
Utilized a 3D-printed robotic hand for testing and improvement.
Developed a motion capture system using MediaPipe's HandLandmarker tailored with a custom glove.
Constructed a dataset linking joint angles to actuator signals via Python-Arduino communication.
Trained various machine learning models, including Linear Regression, SVM, and MLP with CNN1D layers.
Conducted hyperparameter tuning using grid search and tested models' performance across different angles.
Demonstrated the ability to control a robotic hand mimicking human movements effectively.
Showed that precise dynamic equations are unnecessary for successful human-like control of robotic hands.
Indicated strong generalization of models across different hand angles and perspectives.

Abstract

Classical robotic control, particularly the control of a robotic hand, requires precise mathematical modelling of the system’s dynamic equations, making the process complex and computationally demanding. Instead of this approach, this thesis explores the use of deep learning methods for controlling a robotic hand, eliminating the need for exact physical modelling. To this end, a 3D-printed robotic hand is utilized, with improvements made to its functionality and appearance. A motion capture system is developed using MediaPipe’s HandLandmarker, which, although designed for human hands, is adapted through a specially designed glove to enhance joint recognition accuracy. Furthermore, a dataset is constructed, containing both joint angles and the corresponding signals sent to the robotic hand’s actuators via Python-Arduino communication. This dataset is used for training and evaluating various machine learning models, including Linear Regression, SVM, and MLP with CNN1D layers. Hyperparameter tuning is performed through grid search, and the models' generalization across different hand angles and perspectives is tested. The results demonstrate that it is feasible to develop a controller that mimics human hand movements without requiring precise knowledge of the system’s dynamic equations, highlighting the potential of machine learning in robotic control systems.

From Video to Torque: A Deep Learning approach to Robotic Hand Control

Key Points

Abstract

Cite This Study