In traffic signal control (TSC), real-time adaptation to fluctuating traffic patterns (TPs) is crucial for efficiency and safety. Traditional fixed-time and actuated schemes struggle under non-stationary demand, incidents, and the growing heterogeneity of urban traffic. While reinforcement learning (RL) offers powerful controllers, many RL approaches rely on fixed contexts or a single monolithic model, limiting learning efficiency and deployment in large networks. Consequently, this paper proposes a modular, online framework that (i) clusters traffic patterns online into a small set of clusters, (ii) trains cluster-specific RL agents online using cluster-specific simulation models, (iii) initializes new cluster agents from the most similar existing cluster to accelerate learning, and (iv) merges the most similar clusters when the maximum number of clusters is exceeded, retaining the agent that best covers the traffic space. The similarity metric used to compare TP trajectories and the simulation model used for RL training are applied to the same TP trajectories, ensuring consistent measurement and environment parametrization across clustering, training and real-time inference. The result is an adaptive TP-RLTSC system that preserves accuracy while reducing computational burden and time-to-real-world deployment. The approach builds on adaptive two-scale ideas where macro-evolution guides micro-level control; this philosophy is adapted to online TP clustering and cluster-specific RL. This paper presents a conceptual TP-RL-TSC framework focusing on architecture, design rationale and workflows; empirical evaluation and deployment studies are planned for future work.
Klein et al. (Thu,) studied this question.