Traffic Signal Control and Speed Offset Coordination Using Q-Learning for Arterial Road Networks
Tianchen Yuan, Petros A. Ioannou
TL;DR
The paper tackles the decoupled control of arterial signals and freeway ramps by proposing a Q-learning framework that coordinates a traffic signal control (TSC) agent and a dynamic speed offset (DSO) agent. The two agents jointly optimize cycle lengths, phase splits, offsets, and recommended speeds to reduce arterial travel time and off-ramp/intersection queues, with a network controller enforcing a unified plan via a majority-rule mechanism. Microscopic simulations on a mixed freeway-arterial network show the QL-based arterial control (with and without unification) outperforms fixed-time and MAXBAND under low/moderate demands, and still provides queue-dissipation benefits under high demand, with additional gains when freeway control is integrated. The results demonstrate practical potential for cross-domain coordination in urban networks and highlight the reciprocal benefits between arterial and freeway controls in mitigating congestion and spillback.
Abstract
Arterial traffic interacts with freeway traffic, yet the two are controlled independently. Arterial traffic signals do not take into account freeway traffic and how ramps control ingress traffic and have no control over egress traffic from the freeway. This often results in long queues in either direction that block ramps and spill over to arterial streets or freeway lanes. In this paper, we propose an adaptive arterial traffic control strategy that combines traffic signal control (TSC) and dynamic speed offset (DSO) coordination using a Q-learning algorithm for a traffic network that involves a freeway segment and adjacent arterial streets. The TSC agent computes the signal cycle length and split based on observed intersection demands and adjacent freeway off-ramp queues. The DSO agent computes the relative offset and the recommended speeds of both ways between consecutive intersections based on their physical distance, intersection queues, and signal cycles. We evaluate the performance of the proposed arterial traffic control strategy using microscopic traffic simulations of an arterial corridor with seven intersections near the I-710 freeway. The proposed QL-based control significantly outperforms a fixed-time control and MAXBAND in terms of the travel time and the number of stops under low or moderate demands. In high-demand scenarios, the travel-time benefit provided by the QL-based control is reduced as it mitigates off-ramp and intersection queues, which is a necessary trade-off in our perspective. In addition, mutual benefit is obtained by implementing freeway and arterial traffic control simultaneously.