Brain-Inspired Cognitive Model With Attention for Self-Driving Cars

The perception-driven approach and end-to-end system are two major vision-based frameworks for self-driving cars. However, it is difficult to introduce attention and historical information into the autonomous driving process, which are essential for achieving human-like driving in these two methods. In this paper, we propose a novel model for self-driving cars called the brain-inspired cognitive model with attention. This model comprises three parts: 1) a convolutional neural network for simulating the human visual cortex; 2) a cognitive map to describe the relationships between objects in a complex traffic scene; and 3) a recurrent neural network, which is combined with the real-time updated cognitive map to implement the attention mechanism and long-short term memory. An advantage of our model is that it can accurately solve three tasks simultaneously: 1) detecting the free space and boundaries for the current and adjacent lanes; 2) estimating the distances to obstacles and vehicle attitude; and 3) learning the driving behavior and decision-making process of a human driver. Importantly, the proposed model can accept external navigation instructions during an end-to-end driving process. To evaluate the model, we built a large-scale road-vehicle dataset containing over 40000 labeled road images captured by three cameras placed on our self-driving car. Moreover, human driving activities and vehicle states were recorded at the same time.