ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning

TL;DR

This work tackles the challenge of generalizing traffic signal control policies across intersections with diverse structures using reinforcement learning. It introduces ADLight, a universal model that uses movement-based state representations and a set-current-phase-duration action to preserve phase structure, augmented by a movement shuffle data augmentation to improve cross-intersection transfer. The approach demonstrates strong generalization across 11 intersections (8 for training, 3 unseen) with substantial training-time savings (over 80%) and only a modest increase in average waiting time on unseen intersections, especially when augmentation is applied. These results suggest that scalable deployment of RL-based traffic signal control is feasible with cross-structure transfer, enabling efficient adaptation to new junctions without retraining from scratch.

Abstract

Traffic signal control has the potential to reduce congestion in dynamic networks. Recent studies show that traffic signal control with reinforcement learning (RL) methods can significantly reduce the average waiting time. However, a shortcoming of existing methods is that they require model retraining for new intersections with different structures. In this paper, we propose a novel reinforcement learning approach with augmented data (ADLight) to train a universal model for intersections with different structures. We propose a new agent design incorporating features on movements and actions with set current phase duration to allow the generalized model to have the same structure for different intersections. A new data augmentation method named \textit{movement shuffle} is developed to improve the generalization performance. We also test the universal model with new intersections in Simulation of Urban MObility (SUMO). The results show that the performance of our approach is close to the models trained in a single environment directly (only a 5% loss of average waiting time), and we can reduce more than 80% of training time, which saves a lot of computational resources in scalable operations of traffic lights.

Figures (6)

• Figure 1: A standard 4-way intersection with its movement signals and phases.
• Figure 2: The overall structure of ADLight.
• Figure 3: A 3-way intersection with its state.
• Figure 4: A 3-way intersection and the variations based on rotation. Ideally, an RL agent should output the same action for all these cases.
• Figure 5: 11 intersections with different topologies and phases in SUMO.