Deep Learning-Based Automatic Pipeline for 3D Needle Localization on Intra-Procedural 3D MRI

Needle localization in 3D images during MRI-guided interventions is challenging due to the complex structure of biological tissue and the variability in the appearance of needle features in in-vivo MR images. Deep learning networks such as the Mask Regional Convolutional Neural Network (R-CNN) could address this challenge by providing accurate needle feature segmentation in intra-procedural MR images. This work developed an automatic coarse-to-fine pipeline that combines 2.5D and 2D Mask R-CNN to leverage inter-slice information and localize the needle tip and axis in in-vivo intra-procedural 3D MR images.