Pd36-10 Artificial Intelligence Based Real-Time Segmentation and Feature Tracking in Urologic Robotic Assisted Surgery: Preliminary Results and Future Prospects

You have accessJournal of UrologySurgical Technology & Simulation: Artificial Intelligence III (PD36)1 May 2024PD36-10 ARTIFICIAL INTELLIGENCE BASED REAL-TIME SEGMENTATION AND FEATURE TRACKING IN UROLOGIC ROBOTIC ASSISTED SURGERY: PRELIMINARY RESULTS AND FUTURE PROSPECTS Rebecca Canneto, Luca A. Morgantini, Rogerio Garcia Nespolo, Yanneck I. Leiderman, and Simone Crivellaro Rebecca CannetoRebecca Canneto , Luca A. MorgantiniLuca A. Morgantini , Rogerio Garcia NespoloRogerio Garcia Nespolo , Yanneck I. LeidermanYanneck I. Leiderman , and Simone CrivellaroSimone Crivellaro View All Author Informationhttps://doi.org/10.1097/01.JU.0001008916.72488.6a.10AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Challenges in anatomical landmarks recognition and spatial orientation still exist in robotic assisted surgery (RAS). Despite numerous studies using artificial intelligence (AI), lack of its integration in real-time during live surgeries is still a limitation to its use. We propose the adaptation and training of an AI neural network to achieve real-time video segmentation in urologic RAS, to provide surgical guidance and improve RAS training with live video-in-video integration. METHODS: Retrospective single-centre analysis of 15 Single-port retroperitoneal partial and radical nephrectomies. A dataset of 150 surgical image frames was selected and annotated using the Computer Vision Annotation Tool platform (Computer Vision Annotation Tool 3D, Intel Corp). Annotated features consisted of renal artery, renal vein, inferior vena cava, kidney, ureter, psoas muscle, bipolar forceps, monopolar scissors, Cadiere forceps and bulldog clamp. An instance segmentation fully convolutional model based on YOLACT++ and combining a ResNet-50 convolutional neural network with a feature pyramid network, a prediction head network, and a prototype generator network was trained. The model output was overlaid on the original frames displaying mask, location, label and confidence score of the segmented features. Feature detection and segmentation were evaluated calculating the area under the precision-recall curve (AUPR). RESULTS: An example of the model output is in Figure 1. AUPR values for detection and segmentation of each feature are summarized in Table 1. CONCLUSIONS: The neural network can detect and segment features in retroperitoneal RAS videos. We propose a framework that could be integrated in real-time during RAS to guide decision-making and to improve training. Further research with a greater study population will be conducted to assess model accuracy, feasibility of framework application in real-time and feature tracking. Download PPT Source of Funding: None © 2024 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 211Issue 5SMay 2024Page: e796 Advertisement Copyright & Permissions© 2024 by American Urological Association Education and Research, Inc.Metrics Author Information Rebecca Canneto More articles by this author Luca A. Morgantini More articles by this author Rogerio Garcia Nespolo More articles by this author Yanneck I. Leiderman More articles by this author Simone Crivellaro More articles by this author Expand All Advertisement PDF downloadLoading ...