Reinforcement Learning for Robot Assisted Live Ultrasound Examination

Due to its portability, non-invasiveness, and real-time capabilities, ultrasound imaging has been widely adopted for liver disease detection. However, conventional ultrasound examinations heavily rely on operator expertise, leading to high workload and inconsistent imaging quality. To address these challenges, we propose a Robotic Ultrasound Scanning System (RUSS) based on reinforcement learning to automate the localization of standard liver planes. And it can help reduce physician burden while improve scanning efficiency and accuracy. The reinforcement learning agent employs a Deep Q-Network (DQN) integrated with LSTM to control probe movements within a discrete action space, utilizing the cross-sectional area of the abdominal aorta region as the criterion for standard plane determination. Experimental results demonstrate that the system successfully obtained the target plane in three real-world trials. The Peak Signal-to-Noise Ratio (PSNR) is used to evaluate the image similarity and the average value is 21.53 dB, which verifies the effectiveness of the proposed method.