Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DataLight: Offline Data-Driven Traffic Signal Control

Liang Zhang, Yutong Zhang, Jianming Deng, Chen Li

TL;DR

DataLight tackles traffic signal control by learning offline from pre-collected data, using velocity-based state representations and spatial segmentation with self-attention to capture urban traffic dynamics. It combines TD, eigensubspace regularization, and conservative Q-learning losses to learn robust policies without online exploration. Empirical results on CityFlow across multiple real-world datasets show DataLight outperforms state-of-the-art online and offline baselines and demonstrates strong robustness under limited data and COD scenarios. The work highlights the practicality of offline data-driven TSC and provides open-source code for replication and further research.

Abstract

Reinforcement learning (RL) has emerged as a promising solution for addressing traffic signal control (TSC) challenges. While most RL-based TSC systems typically employ an online approach, facilitating frequent active interaction with the environment, learning such strategies in the real world is impractical due to safety and risk concerns. To tackle these challenges, this study introduces an innovative offline data-driven approach, called DataLight. DataLight employs effective state representations and reward function by capturing vehicular speed information within the environment. It then segments roads to capture spatial information and further enhances the spatially segmented state representations with sequential modeling. The experimental results demonstrate the effectiveness of DataLight, showcasing superior performance compared to both state-of-the-art online and offline TSC methods. Additionally, DataLight exhibits robust learning capabilities concerning real-world deployment issues. The code is available at https://github.com/LiangZhang1996/DataLight.

DataLight: Offline Data-Driven Traffic Signal Control

TL;DR

DataLight tackles traffic signal control by learning offline from pre-collected data, using velocity-based state representations and spatial segmentation with self-attention to capture urban traffic dynamics. It combines TD, eigensubspace regularization, and conservative Q-learning losses to learn robust policies without online exploration. Empirical results on CityFlow across multiple real-world datasets show DataLight outperforms state-of-the-art online and offline baselines and demonstrates strong robustness under limited data and COD scenarios. The work highlights the practicality of offline data-driven TSC and provides open-source code for replication and further research.

Abstract

Reinforcement learning (RL) has emerged as a promising solution for addressing traffic signal control (TSC) challenges. While most RL-based TSC systems typically employ an online approach, facilitating frequent active interaction with the environment, learning such strategies in the real world is impractical due to safety and risk concerns. To tackle these challenges, this study introduces an innovative offline data-driven approach, called DataLight. DataLight employs effective state representations and reward function by capturing vehicular speed information within the environment. It then segments roads to capture spatial information and further enhances the spatially segmented state representations with sequential modeling. The experimental results demonstrate the effectiveness of DataLight, showcasing superior performance compared to both state-of-the-art online and offline TSC methods. Additionally, DataLight exhibits robust learning capabilities concerning real-world deployment issues. The code is available at https://github.com/LiangZhang1996/DataLight.
Paper Structure (41 sections, 9 equations, 3 figures, 11 tables)

This paper contains 41 sections, 9 equations, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional TSC Methods
  4. Online RL-based TSC Methods
  5. Offline RL Algorithms
  6. Preliminary
  7. Traffic signal control.
  8. Offline RL.
  9. Problem definition.
  10. DataLight
  11. State and Reward Design
  12. State representations.
  13. Reward function.
  14. Action space.
  15. Network Structure Design
  16. ...and 26 more sections

Figures (3)

  • Figure 1: Illustration of intersection structure and traffic signal phases: (a) A standard four-way, three-lane intersection. (b) Eight distinct traffic movements. (c) Four signal phases.
  • Figure 2: Performance evaluation of DataLight with varying amounts of offline data (ATT in seconds).
  • Figure 3: Performance of DataLight on COD (ATT in seconds).