Dynamic path planning of autonomous mobile robot in off-road environments using experience replay enhanced distributed proximal policy optimization algorithm

In off-road environments, some problems like insufficient consideration of environmental trafficability, difficulty in model convergence, and limited generalization often exist in autonomous mobile robot (AMR) path planning research based on reinforcement learning (RL). In order to solve these problems, a distributed proximal policy optimization dynamic path planning algorithm based on experience playback enhancement (Re-DPPO) is proposed, which realizes the safe, feasible, and optimal trafficability path planning of AMR between any starting and ending points in an off-road environment. First, in order to achieve AMR path planning that takes into account environmental trafficability in off-road environments, an AMR trafficability map that integrates multiple environmental factors was constructed. Second, to ensure that AMR can simultaneously consider path trafficability during real-time obstacle avoidance, a multi-dimensional comprehensive reward function was designed that integrates dynamic obstacle avoidance, goal point proximity, and trafficability evaluation. In addition, in response to the problems of difficult model convergence and low utilization of high-value samples in traditional deep reinforcement learning (DRL) algorithms, a distributed training architecture and a priority experience replay mechanism are introduced. Finally, comparative experiments were conducted between the Re-DPPO algorithm and five DRL algorithms, namely the proximal policy optimization (PPO), experience replay enhanced proximal policy optimization algorithm (Re-PPO), distributed proximal policy optimization (DPPO), deep Q-network (DQN), and double DQN, and the path planning test was carried out in four new scenarios. The results show that among the six algorithms, the Re-DPPO algorithm exhibits the best convergence performance and path planning success rate. This algorithm can prioritize planning the optimal trafficability path while ensuring accessibility, and demonstrates good generalization ability in all four new scenarios.