Scheduling Eight-Phase Urban Traffic Light Problems via Ensemble Meta-Heuristics and Q-Learning Based Local Search

This paper addresses urban traffic light scheduling problems (UTLSP) with eight phases. The objective is to minimize the total vehicle delay time by assigning traffic phases and phase-timing optimally. A novel hybrid algorithm framework by combining meta-heuristics with Q-learning is proposed to solve the UTLSP for the first time. First, a mathematical model is developed to describe UTLSP. Second, five meta-heuristics are employed and improved to solve the concerned problems. Based on the feature of UTLSP, five local search operators are developed to improve the exploitation performance of the meta-heuristics. Third, two Q-learning-based ensemble strategies are designed to select the premium local search operators during the meta-heuristics’ iterations. Finally, experiments are conducted on 10 cases with different scales. A total of 26 algorithms are compared for validation. Experimental results verify the effectiveness of the proposed ensemble strategies. Comparisons and discussions show that the improved water cycle algorithm with the first Q-learning strategy has the best competitiveness for solving the considered problems.