Deep-Reinforcement-Learning-Based Dynamic Navigation for a Mecanum-Wheeled Mobile Robot

Deep reinforcement learning (DRL) has emerged as a key technology to solve navigation tasks of an autonomous mobile robot (AMR) in dynamic environments due to its adaptability to complex decision-making problems. This study aims to develop a DRL method for the dynamic navigation of a Mecanum-wheel AMR. The DRL training architecture is based on the navigation problem of the AMR with continuous states to adopt Soft Actor–Critic (SAC) as the main learning method. To overcome the limitations of the DRL method in the long-range navigation and scene adaptability, the trained SAC model is integrated into the Nav2 framework and combined with the NavFn global planner. In the proposed architecture, the global planner is responsible for path generation and the DRL model focuses on local dynamic obstacle avoidance. Finally, the proposed DRL method was compared with the Dynamic Window Based (DWB) local planner and the Model Predictive Path Integral (MPPI) controller, in terms of the dynamic obstacle avoidance performance, cross-scene adaptability, and parameter tuning requirements, to verify its feasibility and effectiveness.