Reinforcement Learning for Robot Assisted Live Ultrasound Examination

Due to its portability, non-invasiveness, and real-time capabilities, ultrasound imaging has been widely adopted for liver disease detection. However, conventional ultrasound examinations heavily rely on operator expertise, leading to high workload and inconsistent imaging quality. To address these challenges, we propose a Robotic Ultrasound Scanning System (RUSS) based on reinforcement learning to automate the localization of standard liver planes. It can help reduce physician burden while improving scanning efficiency and accuracy. The reinforcement learning agent employs a Deep Q-Network (DQN) integrated with LSTM to control probe movements within a discrete action space, utilizing the cross-sectional area of the abdominal aorta region as the criterion for standard plane determination. System performance was comprehensively evaluated against a target standard plane, achieving an average Peak Signal-to-Noise Ratio (PSNR) of 24.51 dB and a Structural Similarity Index (SSIM) of 0.70, indicating high fidelity in the acquired images. Furthermore, a mean Dice coefficient of 0.80 for the abdominal aorta segmentation confirmed high anatomical localization accuracy. These preliminary results demonstrate the potential of our method for achieving consistent and autonomous ultrasound scanning.