SimPS-Net: Simultaneous Pose & Segmentation Network of Surgical Tools

The ability to detect and localise surgical tools using RGB cameras during robotic assisted surgery can allow for the development of various implementations, such as vision- based active constraints and refinements in robot path planning, which can ultimately lead in improved patient safety during operation. For this purpose, the proposed network, SimPS-Net capable of both detection and 3D pose estimation of standard surgical tools using a single RGB camera, is introduced. In addition to the network, a novel dataset generated for training and testing is presented. The proposed network achieved a mean DICE coefficient of 85.0%, while also exhibiting a low average error of 5.5mm and 3.3◦ for 3D position and orientation respectively, thus outperforming the competing networks.