Multi-Robot Navigation System Design Based on Proximal Policy Optimization Algorithm

The more path conflicts between multiple robots, the more time it takes to avoid each other, and the more navigation time it takes for the robots to complete all tasks. This study designs a multi-robot navigation system based on deep reinforcement learning to provide an innovative and effective method for global path planning of multi-robot navigation. It can plan paths with fewer path conflicts for all robots so that the overall navigation time for the robots to complete all tasks can be reduced. Compared with existing methods of global path planning for multi-robot navigation, this study proposes new perspectives and methods. It emphasizes reducing the number of path conflicts first to reduce the overall navigation time. The system consists of a localization unit, an environment map unit, a path planning unit, and an environment monitoring unit, which provides functions for calculating robot coordinates, generating preselected paths, selecting optimal path combinations, robot navigation, and environment monitoring. We use topological maps to simplify the map representation for multi-robot path planning so that the proposed method can perform path planning for more robots in more complex environments. The proximal policy optimization (PPO) is used as the algorithm for deep reinforcement learning. This study combines the path selection method of deep reinforcement learning with the A* algorithm, which effectively reduces the number of path conflicts in multi-robot path planning and improves the overall navigation time. In addition, we used the reciprocal velocity obstacles algorithm for local path planning in the robot, combined with the proposed global path planning method, to achieve complete and effective multi-robot navigation. Some simulation results in NVIDIA Isaac Sim show that for 1000 multi-robot navigation tasks, the maximum number of path conflicts that can be reduced is 60,375 under nine simulation conditions.