Utilizing proximal policy optimization-based multi-agent reinforcement learning to enhance traffic performance and eco-diving in urban areas

Connected and automated vehicles are expected to play a key role in alleviating urban traffic congestion in transportation systems. However, coordinating traffic lights and connected and automated vehicles remains a challenging problem in mixed-autonomy environments. This paper proposes a modified proximal policy optimization–based decentralized multi-agent reinforcement learning framework to jointly optimize traffic signal control and eco-driving behavior. The model includes two types of agents, namely traffic light agents and connected and automated vehicle agents. Traffic light agents aim to improve traffic efficiency by optimizing signal-phase decisions. Connected and automated vehicle agents focus on adjusting vehicle acceleration and speed to enhance driving comfort and smoothness. Separate policy networks are designed for each agent type to capture their different control objectives. The proposed framework is evaluated on a realistic urban traffic network. Simulation results show that the decentralized model (multi-agent-proximal policy optimization) achieves the best overall performance. It significantly improves average speed compared to baseline methods. It also reduces average travel time and vehicle waiting time. Further analysis shows that increasing connected and automated vehicle penetration improves overall traffic efficiency. The best performance is achieved when all vehicles are connected and automated vehicles, indicating full system observability and coordination.