Learning to Register Needles in Ultrasound Images During Tissue Insertion

such as blood drawing, endovascular device insertion, and biopsy. However, freehand needle insertion to deep targets risks damaging critical anatomic structures. Robotic assistance can help guide the needle to reach its target while avoiding these structures, especially in scenarios where skilled specialists may not be readily available, e.g. rural areas and battlefields. Robotic assis- tance requires real-time imaging and computer vision to monitor the needle’s shape and position during insertion. Compared to options like MRI and CT, point-of-care ultrasound imaging is especially appropriate for this task due to its benefits of real-time video, portability, low cost, and lack of ionizing radiation. Despite these benefits, ultrasound suffers from poor image quality, making it difficult for both humans and computer vision algorithms to track the needle. Needle bending, partially due to interference from heteroge- neous tissue layers, is another challenge. These chal- lenges motivate the need for an effective deformable registration method for needle tracking in ultrasound. Classical methods address deformable registration through an optimization approach. Despite early suc- cesses with these methods, they can have poor accuracy, be computationally costly, and involve many tuning parameters. Recently, deep learning methods have been embraced to address these limitations. One approach, upon which we build our present work, is to represent deformations as optical flow and train a neural network to predict the optical flow vector field [1], [2]. Tradition- ally this requires ground-truth flow maps for regression, but the scarcity of ground-truth data in medical imaging poses a significant hurdle to training these models. To address these issues, we present an unsupervised learning-based approach to deformable registration for needle tracking in ultrasound. In our prior work, we introduced U-RAFT, a model for deformable registration in vessels [3]. Here, we build upon [3] to calculate flow maps for video sequences of needles, an approach we call Video U-RAFT. We test our approach on medical phantom and live-tissue datasets and demonstrate im- proved accuracy in needle tracking.