Multi-agent Reinforcement Traffic Signal Control based on Interpretable Influence Mechanism and Biased ReLU Approximation

TL;DR

This work addresses cooperative traffic signal control across multi-intersection networks by formulating the problem as a MARL task and introducing an interpretable influence mechanism based on EHHNN. It proposes a PWL-actor-critic framework where both the value and policy functions are approximated with Biased ReLU (BReLU) networks, and uses a centralized critic with decentralized actors to coordinate decisions without a predefined adjacency graph. A key contribution is the EHHNN-based influence module, which provides ANOVA-based input importance and a sparse, interpretable representation of spatiotemporal dependencies, enabling effective coordination. Experimental results on METR-LA forecasting and two synthetic traffic networks show improved predictive accuracy, reduced network-wide delays, and stable traffic flows, highlighting the approach’s scalability and practical impact for intelligent transportation systems.

Abstract

Traffic signal control is important in intelligent transportation system, of which cooperative control is difficult to realize but yet vital. Many methods model multi-intersection traffic networks as grids and address the problem using multi-agent reinforcement learning (RL). Despite these existing studies, there is an opportunity to further enhance our understanding of the connectivity and globality of the traffic networks by capturing the spatiotemporal traffic information with efficient neural networks in deep RL. In this paper, we propose a novel multi-agent actor-critic framework based on an interpretable influence mechanism with a centralized learning and decentralized execution method. Specifically, we first construct an actor-critic framework, for which the piecewise linear neural network (PWLNN), named biased ReLU (BReLU), is used as the function approximator to obtain a more accurate and theoretically grounded approximation. Finally, our proposed framework is validated on two synthetic traffic networks to coordinate signal control between intersections, achieving lower traffic delays across the entire traffic network compared to state-of-the-art (SOTA) performance.

Figures (7)

• Figure 1: A non-Euclidean Traffic Network
• Figure 2: Overview of multi-agent BReLU actor-critic framework.
• Figure 3: Structure of EHH network decomposition
• Figure 4: The METR-LA traffic network
• Figure 5: Synthetic Traffic Networks

Theorems & Definitions (3)

• Remark 1
• Lemma 1
• proof