Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control

Maonan Wang, Xi Xiong, Yuheng Kan, Chengcheng Xu, Man-On Pun

TL;DR

The paper tackles urban traffic congestion by introducing UniTSA, a universal reinforcement learning framework for V2X traffic signal control that generalizes across diverse intersection topologies. It uses a junction-matrix representation of intersection states, coupled with five traffic state augmentation methods and a Keep-or-Change action policy, trained with PPO and further refined for important intersections via LoRA fine-tuning. Empirical results on SUMO show UniTSA delivering shorter waiting times than traditional methods and prior universal models, with strong generalization to unseen intersections and substantial training-time savings through fine-tuning. The work provides a practical, open-source framework for data-efficient, generalizable RL-based TSC in heterogeneous urban networks, advancing deployment potential in real-world V2X systems.

Abstract

Traffic congestion is a persistent problem in urban areas, which calls for the development of effective traffic signal control (TSC) systems. While existing Reinforcement Learning (RL)-based methods have shown promising performance in optimizing TSC, it is challenging to generalize these methods across intersections of different structures. In this work, a universal RL-based TSC framework is proposed for Vehicle-to-Everything (V2X) environments. The proposed framework introduces a novel agent design that incorporates a junction matrix to characterize intersection states, making the proposed model applicable to diverse intersections. To equip the proposed RL-based framework with enhanced capability of handling various intersection structures, novel traffic state augmentation methods are tailor-made for signal light control systems. Finally, extensive experimental results derived from multiple intersection configurations confirm the effectiveness of the proposed framework. The source code in this work is available at https://github.com/wmn7/Universal_Light

UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control

TL;DR

The paper tackles urban traffic congestion by introducing UniTSA, a universal reinforcement learning framework for V2X traffic signal control that generalizes across diverse intersection topologies. It uses a junction-matrix representation of intersection states, coupled with five traffic state augmentation methods and a Keep-or-Change action policy, trained with PPO and further refined for important intersections via LoRA fine-tuning. Empirical results on SUMO show UniTSA delivering shorter waiting times than traditional methods and prior universal models, with strong generalization to unseen intersections and substantial training-time savings through fine-tuning. The work provides a practical, open-source framework for data-efficient, generalizable RL-based TSC in heterogeneous urban networks, advancing deployment potential in real-world V2X systems.

Abstract

Traffic congestion is a persistent problem in urban areas, which calls for the development of effective traffic signal control (TSC) systems. While existing Reinforcement Learning (RL)-based methods have shown promising performance in optimizing TSC, it is challenging to generalize these methods across intersections of different structures. In this work, a universal RL-based TSC framework is proposed for Vehicle-to-Everything (V2X) environments. The proposed framework introduces a novel agent design that incorporates a junction matrix to characterize intersection states, making the proposed model applicable to diverse intersections. To equip the proposed RL-based framework with enhanced capability of handling various intersection structures, novel traffic state augmentation methods are tailor-made for signal light control systems. Finally, extensive experimental results derived from multiple intersection configurations confirm the effectiveness of the proposed framework. The source code in this work is available at https://github.com/wmn7/Universal_Light
Paper Structure (18 sections, 27 equations, 9 figures, 5 tables)

This paper contains 18 sections, 27 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Methodology
  5. Framework
  6. Agent Design and Junction Matrix
  7. Traffic State Augmentation
  8. Intersection Feature Extraction
  9. RL Training and Fine-tuning
  10. Experiments
  11. Experiment Settings
  12. Datasets
  13. Compared Methods
  14. Results of the training intersections
  15. Results of the test intersections
  16. ...and 3 more sections

Figures (9)

  • Figure 1: A standard $4$-way intersection illustration.
  • Figure 2: The overall framework of UniTSA.
  • Figure 3: (a) A $3$-way intersection with (b) its junction matrix with zero padding.
  • Figure 4: Illustration of three traffic state augmentation methods applied to both $4$-way and $3$-way intersections.
  • Figure 5: Detailed steps in traffic state augmentation block.
  • ...and 4 more figures