Improving the Robustness of Surgical Instrument Segmentation Models via GAN-Based Data Augmentation

In robotics-assisted surgery, accurate segmentation of surgical instruments from background pixels is crucial for the success of various downstream tasks. Currently, popular segmentation models such as the U-Net exhibit excellent performance under major surgical conditions. However, when benchmarking the segmentation models under unforeseen surgical complications such as overbleeding, concerns toward their robustness remain. In fact, due to the lack of training datasets for the surgical segmentation models under complications, these models often exhibit degraded performances and thus result in failures for conducting accurate segmentations at the pixel level. In this paper, we leverage the Generative Adversarial Network (GAN), specifically through generating realistic images that simulate the condition of overbleeding to enlarge the training datasets a process known as data augmentation. With more diverse training data from different scenarios, segmentation models can be trained better and reduce the risk of errors. In addition, a dataset consisting of 17 mock endoscopic video sequences with binary masks indicating ground truths was used for training/testing. With the dataset, a U-Net architecture is used to examine performance through metrics such as the Normalized Surface Distance (NSD) and Dice Similarity Coefficient (DSC). As a result, integrations with GAN-based data augmentation allow segmentation models performances to improve 9.07% for NSD and 6.63% for DSC. From the results, our GAN-based augmentation is proven to be effective, providing a novel direction for improving models robustness during the training procedure, paving the way for future exploration and clinical translation of the cutting-edge deep learning models into real robotics surgery.