Motion Invariant and Variant Kinematic Features in Surgical Gesture Recognition

Recognizing surgical gestures in real time is critical for automated activity recognition, skill assessment, and surgical assistance. The current robotic surgical systems provide us with rich multi-modal data such as video and kinematics. While some recent works in multi-modal neural networks learn the relationships between vision and kinematics data, current approaches treat kinematics information as independent signals, with no underlying relation between tool-tip poses. However, instrument poses are geometrically related, and the underlying geometry can aid neural networks in learning gesture representation. Therefore, we propose integrating motion invariant signals, arc length, dual angle, curvature, and torsion, with vision and kinematics using a relational graph network to capture the underlying relations between different data streams. We show that gesture recognition improves when combining motion invariant signals, achieving 92.9% frame-wise accuracy on Suturing. Our results show that motion invariant signals combined with positional data provide more interpretable representations than conventional position and quaternion signals. These findings highlight the value of geometric-aware modeling of kinematics for surgical gesture recognition and suggest that motion invariant signals can improve generalization across users.