Robots that operate in static environments with fixed obstacles are crucial for automating processes in industries such as manufacturing, warehousing, and healthcare. Although these environments are static, designing a navigation algorithm that achieves high accuracy, reasonable efficiency, and low computational complexity remains a significant challenge. Besides performing well in familiar settings, the algorithm must also be capable of generalizing to other static environments with different obstacle configurations.To address these challenges, this paper presents a navigation algorithm based on deep reinforcement learning (DRL), specifically a combination of double Q-networks, multi-stage learning, and priority experience replay (PER). This algorithm is referred to as PER-MS-DDQN. The key innovation of this research is the simultaneous development of a new reward function derived from the A* algorithm and a new definition of robot states. This combination enhances navigation accuracy, speeds up the learning process, and mitigates the issue of scattered rewards. The proposed algorithm was initially trained in a basic static environment using Grid World and subsequently evaluated both in this familiar setting and in larger, more complex static environments, including the ROS2 software environment. Additionally, the algorithm was tested in a real-world environment. The results indicate that over 500 episodes, the robot achieved a success rate of 99% in reaching its goals. Moreover, there was a significant improvement in both accuracy and navigation time compared to previous methods. These findings demonstrate the practicality and efficiency of the proposed algorithm and its ability to generalize in real-world scenarios.
Korayem et al. (Sun,) studied this question.