Human-like Autonomous Vehicle Speed Control by Deep Reinforcement Learning with Double Q-Learning

Autonomous driving has become a popular research project. How to control vehicle speed is a core problem in autonomous driving. Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to control the vehicle speed. However, the popular Q-learning algorithm is unstable in some games in the Atari 2600 domain. In this paper, a reinforcement learning approach called Double Q-learning is used to control a vehicle's speed based on the environment constructed by naturalistic driving data. Depending on the concept of the direct perception approach, we propose a new method called integrated perception approach to construct the environment. The input of the model is made up of high dimensional data including road information processed from the video data and the low dimensional data processed from the sensors. During experiment, compared with deep Q-learning algorithm, double deep Q-learning has improvements both in terms of value accuracy and policy quality. Our model's score is 271.73% times that of deep Q-learning.