Robot-Assisted Training in Laparoscopy Using Deep Reinforcement Learning

Minimally invasive surgery (MIS) is increasingly becoming a vital method of reducing surgical trauma and significantly improving postoperative recovery. However, skillful handling of surgical instruments used in MIS, especially for laparoscopy, requires a long period of training and depends highly on the experience of surgeons. This letter presents a new robot-assisted surgical training system which is designed to improve the practical skills of surgeons through intrapractice feedback and demonstration from both human experts and reinforcement learning (RL) agents. This system utilizes proximal policy optimization to learn the control policy in simulation. Subsequently, a generative adversarial imitation learning agent is trained based on both expert demonstrations and learned policies in simulation. This agent then generates demonstration policies on the robot-assisted device for trainees and produces feedback scores during practice. To further acquire surgical tools coordinates and encourage self-oriented practice, a mask region-based convolution neural network is trained to perform the semantic segmentation of surgical tools and targets. To the best of our knowledge, this system is the first robot-assisted laparoscopy training system which utilizes actual surgical tools and leverages deep reinforcement learning to provide demonstration training from both human expert perspectives and RL criterion.