Gesture Recognition for Shared Control in Robotic Suturing with LSTM Networks and Transformers

Surgical procedures are characterised by specific move- ments of the surgeon’s hands to complete tasks such as suturing, cutting, or object relocation. Understanding the context and kinematics of these procedures can allow us to predict the goal that the surgeon is trying to achieve and provide the correct external guidance. In this paper, we develop a real-time surgical gesture recognition method for shared control in robotic sur- gical tasks. We implement Long Short-Term Memory (LSTM) and Transformers-based methods to verify their performance for real-time applications. To prove the effectiveness of our methods, we evaluate and assess it on an existing dataset. We demonstrate that a high performance can be achieved for real-time recognition of kinematic surgical gestures by categorising the suturing task in four essential surgeme motions (Figure 1) and one for general movements.