Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems

Xingchen Zou, Yuhao Yang, Zheng Chen, Xixuan Hao, Yiqi Chen, Chao Huang, Yuxuan Liang

TL;DR

Traffic-R1 presents a lightweight 3B-parameter reinforced LLM designed for traffic signal control that achieves zero-shot generalization, edge-compatible inference, and human-like, explainable reasoning. The method uses a two-stage RL pipeline—offline expert-informed training and online open-world exploration—coupled with an asynchronous communication network to coordinate multiple intersections. An expert-collaborative dataset supports offline policy optimization via Group Relative Policy Optimization, while online STPO with group advantages enables dense, multi-intersection learning. Real-world deployment in a city demonstrates notable reductions in average queue lengths and operator workload, underscoring the practical viability and scalability of reinforced LLMs for urban traffic management.

Abstract

We introduce Traffic-R1, a 3B-parameter foundation model with human-like reasoning for Traffic signal control (TSC), developed via self-exploration and iterative reinforcement of LLM with expert guidance in a simulated traffic environment. Compared with traditional reinforcement learning and recent LLM-based methods, Traffic-R1 offers three main advantages: zero-shot generalization, transferring unchanged to new road networks and out-of-distribution incidents by leveraging internal traffic-control policies and reasoning; a compact 3B-parameter design that supports real-time inference on mobile-class chips for edge deployment; and an explainable TSC process that enables multi-intersection coordination through communication and an asynchronous communication network. Extensive benchmarks show Traffic-R1 outperforms strong baselines and training-intensive RL controllers. In production, the model now manages signals affecting over 55,000 drivers daily, reduces average queue lengths by more than 5%, and halves operator workload. Our model is available at https://huggingface.co/Season998/Traffic-R1.

Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems

TL;DR

Traffic-R1 presents a lightweight 3B-parameter reinforced LLM designed for traffic signal control that achieves zero-shot generalization, edge-compatible inference, and human-like, explainable reasoning. The method uses a two-stage RL pipeline—offline expert-informed training and online open-world exploration—coupled with an asynchronous communication network to coordinate multiple intersections. An expert-collaborative dataset supports offline policy optimization via Group Relative Policy Optimization, while online STPO with group advantages enables dense, multi-intersection learning. Real-world deployment in a city demonstrates notable reductions in average queue lengths and operator workload, underscoring the practical viability and scalability of reinforced LLMs for urban traffic management.

Abstract

We introduce Traffic-R1, a 3B-parameter foundation model with human-like reasoning for Traffic signal control (TSC), developed via self-exploration and iterative reinforcement of LLM with expert guidance in a simulated traffic environment. Compared with traditional reinforcement learning and recent LLM-based methods, Traffic-R1 offers three main advantages: zero-shot generalization, transferring unchanged to new road networks and out-of-distribution incidents by leveraging internal traffic-control policies and reasoning; a compact 3B-parameter design that supports real-time inference on mobile-class chips for edge deployment; and an explainable TSC process that enables multi-intersection coordination through communication and an asynchronous communication network. Extensive benchmarks show Traffic-R1 outperforms strong baselines and training-intensive RL controllers. In production, the model now manages signals affecting over 55,000 drivers daily, reduces average queue lengths by more than 5%, and halves operator workload. Our model is available at https://huggingface.co/Season998/Traffic-R1.

Paper Structure

This paper contains 31 sections, 4 equations, 8 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Methodology
  5. Human-Informed RL
  6. Expert-Collaborative TSC Dataset
  7. Offline Policy Optimization
  8. Open-World Reinforcement Learning
  9. Asynchronous Communication Network
  10. Experiments
  11. Experimental Settings
  12. Dataset
  13. Implementation Details
  14. Baseline Methods
  15. Evaluation Protocols
  16. ...and 16 more sections

Figures (8)

  • Figure 1: Introduction of Traffic-R1, a foundation (covering six features) reinforced LLM for TSC systems.
  • Figure 2: Introduction of the two-stage RL framework
  • Figure 3: Expert-Collaborative Dataset Construction
  • Figure 4: Asynchronous communication network compared with conventional synchronized network
  • Figure 5: Comparison results for models' capacities.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.3