RSU-Assisted Traffic-Aware Routing Based on Reinforcement Learning for Urban Vanets

In urban vehicular ad hoc networks (VANETs), the high mobility of vehicles along street roads poses daunting challenges to routing protocols and has a great impact on network performance. In addition, the frequent network partition caused by an uneven distribution of vehicles in an urban environment further places higher requirements on the routing protocols in VANETs. More importantly, the high vehicle density during the traffic peak hours and a variety of natural obstacles, such as tall buildings, other vehicles and trees, greatly increase the difficulty of protocol design for high quality communications. Considering these issues, in this paper, we introduce a novel routing protocol for urban VANETs called RSU-assisted Q-learning-based Traffic-Aware Routing (QTAR). Combining the advantages of geographic routing with the static road map information, QTAR learns the road segment traffic information based on the Q-learning algorithm. In QTAR, a routing path consists of multiple dynamically selected high reliability connection road segments that enable packets to reach their destination effectively. For packet forwarding within a road segment, distributed V2V Q-learning (Q-learning occurs between vehicles) integrated with QGGF (Q-greedy geographical forwarding) is adopted to reduce delivery delay and the effect of fast vehicle movements on path sensitivity, while distributed R2R Q-learning (Q-learning occurs between RSU units) is designed for packet forwarding at each intermediate intersection. In the case of a local optimum occurring in QGGF, SCF (store-carry-forward) is used to reduce the possibility of packet loss. Detailed simulation experimental results demonstrate that QTAR outperforms the existing traffic-aware routing protocols, in terms of 7.9% and 16.38% higher average packet delivery ratios than those of reliable traffic-aware routing (RTAR) and greedy traffic-aware routing (GyTAR) in high vehicular density scenarios and 30.96% and 46.19% lower average end-to-end delays with respect to RTAR and GyTAR in low vehicular density scenarios, respectively.