An improved proximal policy optimization algorithm for autonomous driving decision-making

In recent years, there has been significant development in research on methods for autonomous driving decision-making based on deep reinforcement learning, including Deep Deterministic Policy Gradient (DDPG) , Proximal Policy Optimization (PPO) , and others. Reinforcement learning strategies based on proximal policy optimization often suffer from poor performance due to getting trapped in local optima and inefficient learning. To address the challenges of proximal policy optimization in autonomous driving, we propose an autonomous driving decision-making algorithm based on adaptive curiosity mechanism and experience replay. Firstly, we introduce a mechanism for adaptive curiosity adjustment based on episode length to address the issue of local optima in proximal policy optimization. By incorporating curiosity mechanisms, the algorithm's exploratory nature is enhanced. Additionally, to ensure safety, we impose constraints on the curiosity coefficient using variations in episode length. Secondly, to overcome the low sample utilization and inefficient learning in PPO, we combine it with an experience replay mechanism, enabling faster learning of superior policies and improving learning efficiency. In simulation experiments, our algorithm efficiently explores a better and safer driving strategy.