Hybrid Tracking Module for Real-Time Tool Tracking for an Autonomous Exoscope

Exoscopes have emerged as a promising visual solution within the field of microneurosurgery. However, manual repositioning poses a challenge causing interruptions that disrupt the surgical flow. Thus, the need for hands-free exoscope control arises. This paper introduces a position-based visual-servoing control approach, comprising a detection module, a hybrid tracking module, and a control module that adjusts a robotic camera holder to follow a surgical tool. The hybrid module was integrated to track and predict the surgical tool's future position to minimize system latency. The proposed system is composed of a 7 Degree-of-Freedom robotic manipulator with an eye-in-hand stereo camera. A comparative analysis with three alternative approaches (Convolutional Neural Network - CNN, Particle Filter - PF, Optical Flow - OF) was assessed using Tracking Error and Center Error metrics. Results showed improved tracking accuracy with an average error of <tex-math notation="LaTeX">9.84 ± 0.08</tex-math> mm for slow movements (2.5 cm/s) and <tex-math notation="LaTeX">13.11 ± 0.39</tex-math> mm for rapid movements (4 cm/s). Finally, a User Study was conducted to investigate whether the proposed system effectively reduced the users' workload compared to the manual repositioning of the camera.