Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PyTSC: A Unified Platform for Multi-Agent Reinforcement Learning in Traffic Signal Control

Rohit Bokade, Xiaoning Jin

TL;DR

PyTSC is introduced, a robust and flexible simulation environment that facilitates the training and evaluation of MARL algorithms for TSC, and provides new opportunities for advancing intelligent traffic management systems in real-world applications.

Abstract

Multi-Agent Reinforcement Learning (MARL) presents a promising approach for addressing the complexity of Traffic Signal Control (TSC) in urban environments. However, existing platforms for MARL-based TSC research face challenges such as slow simulation speeds and convoluted, difficult-to-maintain codebases. To address these limitations, we introduce PyTSC, a robust and flexible simulation environment that facilitates the training and evaluation of MARL algorithms for TSC. PyTSC integrates multiple simulators, such as SUMO and CityFlow, and offers a streamlined API, empowering researchers to explore a broad spectrum of MARL approaches efficiently. PyTSC accelerates experimentation and provides new opportunities for advancing intelligent traffic management systems in real-world applications.

PyTSC: A Unified Platform for Multi-Agent Reinforcement Learning in Traffic Signal Control

TL;DR

PyTSC is introduced, a robust and flexible simulation environment that facilitates the training and evaluation of MARL algorithms for TSC, and provides new opportunities for advancing intelligent traffic management systems in real-world applications.

Abstract

Multi-Agent Reinforcement Learning (MARL) presents a promising approach for addressing the complexity of Traffic Signal Control (TSC) in urban environments. However, existing platforms for MARL-based TSC research face challenges such as slow simulation speeds and convoluted, difficult-to-maintain codebases. To address these limitations, we introduce PyTSC, a robust and flexible simulation environment that facilitates the training and evaluation of MARL algorithms for TSC. PyTSC integrates multiple simulators, such as SUMO and CityFlow, and offers a streamlined API, empowering researchers to explore a broad spectrum of MARL approaches efficiently. PyTSC accelerates experimentation and provides new opportunities for advancing intelligent traffic management systems in real-world applications.

Paper Structure

This paper contains 28 sections, 1 equation, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Challenges in Current MARL Research for TSC
  3. Contributions
  4. The PyTSC Framework
  5. Support for Diverse Simulator Backends:
  6. Customizable:
  7. Optimized Performance:
  8. Integration of Static Network Features:
  9. Compatibility with MARL Training Frameworks:
  10. Experiments
  11. Traffic Signal Network Scenarios
  12. Traffic Signals as Agents
  13. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs)
  14. Observation representation
  15. Action Representation
  16. ...and 13 more sections

Figures (8)

  • Figure 1: Overview of PyTSC Architecture
  • Figure 2: $2\times2$ Grid SUMO (left) CityFlow (right)
  • Figure 3: $3\times3$ Grid SUMO (left) CityFlow (right)
  • Figure 4: Real-world environments for SUMO
  • Figure 5: Real-world environments for CityFlow
  • ...and 3 more figures