Traffic signal control by distributed Reinforcement Learning with min-sum communication

Reinforcement learning (RL) is a promising technique for adaptive signal control based on real-time traffic data. However, the implementation of RL in large-scale traffic signal network remains an open challenge due to the extremely high computational cost. In this paper, we integrate a distributed communication technique into RL so that the learning and searching cost of large-scale multi-agent system can be significantly reduced. In particular, we first decompose the global Q-function into local Q-functions, so each local signal agent can compute its own optimal policy based on local observations. We then apply the max-sum message passing algorithm to share information among agents, for finding a stable and optimal global policy that is accepted by all agents. Due to this information sharing characteristic, our approach is more optimal than decentralized RL, and it is much more efficient than centralized RL. Further, we perform realistic simulation to demonstrate that this distributed RL approach outperforms decentralized RL algorithm, as well as some typical heuristic decentralized control methods.