Robotic Instrument Segmentation With Image-to-Image Translation

The semantic segmentation of robotic surgery video and the delineation of robotic instruments are important for enabling automation. Despite major recent progresses, the majority of the latest deep learning models for instrument detection and segmentation rely on large datasets with ground truth labels. While demonstrating the capability, reliance on large labelled data is a problem for practical applications because systems would need to be re-trained on domain variations such as procedure type or instrument sets. In this letter, we propose to alleviate this problem by training deep learning models on datasets that are synthesised using image-to-image translation techniques and we investigate different methods to perform this process optimally. Experimentally, we demonstrate that the same deep network architecture for robotic instrument segmentation can be trained on both real data and on our proposed synthetic data without affecting the quality of the output models' performance. We show this for several recent approaches and provide experimental support on publicly available datasets, which highlight the potential value of this approach.