Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs
Kang Wang, Zhishu Shen, Zhen Lei, Tiehua Zhang
TL;DR
The paper tackles urban traffic signal control by capturing higher‑order spatio‑temporal interactions among intersections using a dynamic spatio‑temporal hypergraph within a multi‑agent reinforcement learning framework. It introduces MA‑SAC augmented with a hypergraph learning module that dynamically constructs spatial and temporal hyperedges with master and candidate nodes, and optimizes policies across MEC‑connected intersections. Empirical results on synthetic and real‑world CityFlow data show that HG‑DRL achieves lower average travel time $ATT$ and higher throughput than strong baselines, with improved stability and scalability in large networks. The work advances urban traffic management by combining edge intelligence, hypergraph representations, and entropy‑regularized multi‑agent RL, and provides code to support reproducibility.
Abstract
Traffic signal control systems (TSCSs) are integral to intelligent traffic management, fostering efficient vehicle flow. Traditional approaches often simplify road networks into standard graphs, which results in a failure to consider the dynamic nature of traffic data at neighboring intersections, thereby neglecting higher-order interconnections necessary for real-time control. To address this, we propose a novel TSCS framework to realize intelligent traffic control. This framework collaborates with multiple neighboring edge computing servers to collect traffic information across the road network. To elevate the efficiency of traffic signal control, we have crafted a multi-agent soft actor-critic (MA-SAC) reinforcement learning algorithm. Within this algorithm, individual agents are deployed at each intersection with a mandate to optimize traffic flow across the road network collectively. Furthermore, we introduce hypergraph learning into the critic network of MA-SAC to enable the spatio-temporal interactions from multiple intersections in the road network. This method fuses hypergraph and spatio-temporal graph structures to encode traffic data and capture the complex spatio-temporal correlations between multiple intersections. Our empirical evaluation, tested on varied datasets, demonstrates the superiority of our framework in minimizing average vehicle travel times and sustaining high-throughput performance. This work facilitates the development of more intelligent urban traffic management solutions. We release the code to support the reproducibility of this work at https://github.com/Edun-Eyes/TSC