Pd27-06 3d Rendering of Cystoscopy Video Footage: A Novel Method Utilizing Neural Radiance Field Processing

You have accessJournal of UrologySurgical Technology & Simulation: Artificial Intelligence II (PD27)1 May 2024PD27-06 3D RENDERING OF CYSTOSCOPY VIDEO FOOTAGE: A NOVEL METHOD UTILIZING NEURAL RADIANCE FIELD PROCESSING Jonathan E. Katz, Jamie Finnegan, Jingpei Lu, Shan Lin, Michael Yip, and Roger Sur Jonathan E. KatzJonathan E. Katz , Jamie FinneganJamie Finnegan , Jingpei LuJingpei Lu , Shan LinShan Lin , Michael YipMichael Yip , and Roger SurRoger Sur View All Author Informationhttps://doi.org/10.1097/01.JU.0001008580.58088.27.06AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Recent developments in Neural Radiance Field Processing (NERF) have leveraged the power of neural networks to quickly reconstruct 3D spaces from 2D images. Our objective was to utilize this technology to 3D render video recordings of diagnostic cystoscopies and test their fidelity. METHODS: With IRB approval we recorded two diagnostic cystoscopies (one with an Ambu disposable flexible cystoscope and the other with a Wolf digital cystoscope). We converted the videos to images and then curated the images to choose approximately 100 images, which minimized blur and spanned a large segment of the bladder. We then utilized the NVIDIA Instant Neural Graphics Primitives (iNGP), a NeRF algorithm that uses multiresolution hash encoding with a compact neural network for significantly faster convergence, to reconstruct the bladder and render novel, unseen views. We computed the structural index similarity (SSIM) and Peak signal to noise ratio (PSNR) to assess the quality and fidelity of the 3D rendering. RESULTS: Both videos were able to be utilized for 3D rendering using iNGP. The rendering derived from the Wolf cystoscopy had a PSNR=29.8 [min=27.2, max=32.6] and SSIM=0.89. Similarly, the rendering derived from the Ambu cystoscopy had a PSNR=31.3 [min=27.1, max=35.1] and SSIM=0.90, static sample images below (Image 1). CONCLUSIONS: Independent of cystoscopy equipment, both 3D renderings achieved reasonable fidelity. Major limitations to widespread adoption of this technology include the need for a curator to select representative and high-quality images from the initial cystoscopy video recording and the relatively small segments of bladder successfully rendered. Nonetheless, we feel that with further refinement this technology can be scaled to create 3D renderings of cystoscopies that will enable evaluation of both completeness and quality of the cystoscopy. Furthermore, this technology would be able to facilitate the comparison of cystoscopies performed in the same patient over time. Download PPT Source of Funding: None © 2024 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 211Issue 5SMay 2024Page: e552 Advertisement Copyright & Permissions© 2024 by American Urological Association Education and Research, Inc.Metrics Author Information Jonathan E. Katz More articles by this author Jamie Finnegan More articles by this author Jingpei Lu More articles by this author Shan Lin More articles by this author Michael Yip More articles by this author Roger Sur More articles by this author Expand All Advertisement PDF downloadLoading ...