Spatiotemporal Decision Transformer for Traffic Coordination

TL;DR

MADT reframes multi-agent traffic signal control as conditional sequence modeling by integrating graph attention with a Decision Transformer and return-to-go conditioning. This yields a spatiotemporal architecture that captures both network topology and dynamic traffic flow, enabling offline learning from historical data and potential online fine-tuning. It achieves state-of-the-art results across synthetic grids and real-world Atlanta and Boston networks, with 5.3–5.9% reductions in average travel time and 5.5–6.2% throughput gains over strong baselines, as well as significant improvements in coordination among adjacent intersections. The approach demonstrates robustness to data quality, favorable scaling within tested networks, and interpretable coordination patterns such as green waves and spillback prevention, highlighting practical potential for real-world deployment with careful safety and adaptation considerations.

Abstract

Traffic signal control is a critical challenge in urban transportation, requiring coordination among multiple intersections to optimize network-wide traffic flow. While reinforcement learning has shown promise for adaptive signal control, existing methods struggle with multi-agent coordination and sample efficiency. We introduce MADT (Multi-Agent Decision Transformer), a novel approach that reformulates multi-agent traffic signal control as a sequence modeling problem. MADT extends the Decision Transformer paradigm to multi-agent settings by incorporating: (1) a graph attention mechanism for modeling spatial dependencies between intersections, (2) a|temporal transformer encoder for capturing traffic dynamics, and (3) return-to-go conditioning for target performance specification. Our approach enables offline learning from historical traffic data, with architecture design that facilitates potential online fine-tuning. Experiments on synthetic grid networks and real-world traffic scenarios demonstrate that MADT achieves state-of-the-art performance, reducing average travel time by 5-6% compared to the strongest baseline while exhibiting superior coordination among adjacent intersections.

Figures (6)

• Figure 1: Comparison of approaches for multi-agent traffic signal control. Independent DQN treats each intersection independently with no coordination mechanism. CoLight introduces spatial coordination through graph attention over the road network but lacks temporal sequence modeling. MAT (Multi-Agent Transformer) applies transformer-based sequence modeling with agent-level attention but ignores the spatial structure of traffic networks. MADT (Ours) combines all three elements: graph attention for spatial coordination based on road network topology, temporal transformer encoding for sequence modeling, and return-to-go conditioning for goal-directed behavior. This unified architecture enables both local coordination (through graph attention) and long-horizon planning (through temporal attention).
• Figure 2: MADT architecture overview. The model takes observations from $N$ intersections (queue lengths, waiting times, current phase), return-to-go targets, and previous actions as input. These are processed through an observation encoder with agent embeddings, followed by graph attention layers that capture spatial dependencies based on road network topology. A temporal transformer encoder with causal attention processes the sequence over time. Finally, parallel action heads generate joint actions for all intersections. The graph attention mechanism ensures coordination emerges from spatial proximity in the road network rather than arbitrary agent ordering.
• Figure 3: Training pipeline for MADT. Stage 1 (Data Collection): Trajectories are collected from CityFlow traffic simulator using a MaxPressure policy, generating 1,000 episodes with 720K decision points. Stage 2 (Data Processing): Trajectories are processed to compute return-to-go values $R_t = \sum_{t'=t}^T r_{t'}$, observations are normalized, and sequences of length $K=20$ are created. Stage 3 (Model Training): The MADT model is trained for 100 epochs with batch size 64 using AdamW optimizer and cross-entropy loss. Stage 4 (Evaluation): Trained models are evaluated on 50 test episodes with varied traffic demands and statistical significance testing.
• Figure 4: Real-time inference pipeline for MADT deployment. Real-Time Input: Traffic sensors (loop detectors, cameras, V2I communication) provide observations including queue lengths, waiting times, and current phase at 1 Hz. Return-to-Go Conditioning: Target performance is specified based on desired ATT reduction, with different targets for peak vs. off-peak hours. MADT Inference: The trained model performs graph attention over the intersection network, processes temporal context from the most recent $K=20$ steps, and generates parallel action predictions for all $N$ agents. Forward pass latency is $<$50ms, well within real-time requirements. Action Execution: Phase selections and timing adjustments are sent to the signal controller interface with a 5-second control cycle, and a feedback loop updates sensor observations.
• Figure 5: Real-world evaluation networks. (a) Atlanta, GA: 16-intersection regular grid network in the Midtown district covering Peachtree Street and adjacent arterials. Numbers indicate intersection IDs. (b) Boston, MA: 15-intersection irregular network in the Back Bay district featuring diagonal arterials (Commonwealth Ave, Massachusetts Ave), the Charles River boundary, and heterogeneous intersection connectivity typical of historic urban layouts.

Theorems & Definitions (1)

• proof