DDPG-Based Path Planning for Medical Microrobot Vascular Navigation

This research focuses on the critical challenge of autonomous path planning for medical microrobots in complex and constrained environments, such as targeted drug delivery and minimally invasive surgery. Traditional planning algorithms often struggle with the dynamic nature and stringent safety requirements of medical applications. To address this, we propose a framework based on deep reinforcement learning (DRL) to train an agent capable of navigating from a start to a target point while adhering to strict safety boundaries. The problem is formulated as a Markov Decision Process (MDP), defining the robot's state by its position and orientation. We systematically investigate and compare two prominent DRL algorithms: Deep Q-Networks (DQN) for discrete action spaces and Deep Deterministic Policy Gradient (DDPG) for continuous control, allowing for finer and smoother movements. A key innovation of this work is the design of a composite reward function tailored for medical scenarios. This function incorporates incentives for reaching the target, penalties for boundary violations and deviations from a predefined safety corridor, and considerations for path efficiency and smoothness. This meticulously crafted reward mechanism ensures that the agent prioritizes safety and precision above all else. The models, built upon Multi-Layer Perceptrons, are trained to optimize the long-term cumulative reward, effectively learning a robust policy for safe and efficient navigation in simulated medical environments. This study demonstrates the significant potential of DRL in developing intelligent and reliable guidance systems for next-generation medical robotics.