Deep Reinforcement Learning-Based Autonomous Navigation for Mobile Robots in Dynamic Environments

When dynamic obstacles are present in the environment, traditional navigation methods often struggle to achieve safe and efficient obstacle avoidance due to their lack of real-time adaptability. To address this challenge, we propose an Ac-tion-Constrained Regularized Twin Delayed Deep Deterministic Policy Gradient (ACR-TD3) algorithm. This algorithm introduces action-constrained regularization (ACR) into the framework of the Twin Delayed Deep Deterministic Policy Gradient (TD3) to optimize navigation policies, ensuring that the robot outputs reasonable mo-tion commands and thereby reduces collision frequency, achieving higher navigation success rates. Additionally, we design a multilayer reward function, combined with the ACR, to further optimize navigation performance. Our proposed method does not rely on environmental maps and achieves end-to-end autonomous navigation based solely on LiDAR input. Experimental results demonstrate that ACR-TD3 achieves a 99% navigation success rate in simulated environments, outperforming classical algorithms such as Deep Deterministic Policy Gradient (DDPG), TD3, and Soft Actor-Critic (SAC), while also exhibiting strong generalization capabilities.