Probe positioning for robot-assisted intraoperative ultrasound imaging using deep reinforcement learning

Purpose. Finding desired scan planes in ultrasound (US) imaging is a critical first task that can be time-consuming, influenced by operator experience, and subject to inter-operator variability. To circumvent these problems, interventional US imaging often necessitates dedicated, experienced sonographers in the operating room. This work presents a new approach leveraging deep reinforcement learning (RL) to assist probe positioning. Methods. A deep Q-network (DQN) is applied and evaluated for renal imaging and is tasked with locating the dorsal US scan plane. To circumvent the need for large labeled datasets, images were resliced from a large dataset of CT volumes and synthesized to US images using Field II, CycleGAN, and U-GAT-IT. The algorithm was evaluated on both synthesized and real US images, and its performance was quantified in terms of the agent's accuracy in reaching the target scan plane. Results. Learning-based synthesis methods performed better than the physics-based approach, achieving comparable image quality when qualitatively compared to real US images. The RL agent was successful in reaching target scan planes when adjusting the probe's rotation, with the U-GAT-IT model demonstrating superior generalizability (80.3% reachability) compared to CycleGAN (54.8% reachability). Conclusions. The approach presents a novel RL training strategy using image synthesis for automated US probe positioning. Ongoing efforts aim to evaluate advanced DQN models, image-based reward functions, and support probe motion with higher degrees of freedom.