Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Tuning of Parameterized Traffic Controllers via Multi-Agent Reinforcement Learning

Giray Önür, Azita Dabiri, Bart De Schutter

TL;DR

The paper addresses congestion in transportation networks by integrating reinforcement learning with parameterized state-feedback controllers. It proposes a multi-agent RL framework where each agent tunes its associated controller's parameters at a slower cadence, preserving fast reactive control. The approach offers decentralization, improved training efficiency, and resilience to partial failures, and is evaluated on a multi-class METANET freeway network showing favorable performance against fixed-parameter controls and comparable results to a centralized RL baseline. The results highlight robustness to disturbances and the benefit of continuous-action parameter tuning over discrete parameter sets.

Abstract

Effective traffic control is essential for mitigating congestion in transportation networks. Conventional traffic management strategies, including route guidance, ramp metering, and traffic signal control, often rely on state feedback controllers, used for their simplicity and reactivity; however, they lack the adaptability required to cope with complex and time-varying traffic dynamics. This paper proposes a multi-agent reinforcement learning framework in which each agent adaptively tunes the parameters of a state feedback traffic controller, combining the reactivity of state feedback controllers with the adaptability of reinforcement learning. By tuning parameters at a lower frequency rather than directly determining control actions at a high frequency, the reinforcement learning agents achieve improved training efficiency while maintaining adaptability to varying traffic conditions. The multi-agent structure further enhances system robustness, as local controllers can operate independently in the event of partial failures. The proposed framework is evaluated on a simulated multi-class transportation network under varying traffic conditions. Results show that the proposed multi-agent framework outperforms the no control and fixed-parameter state feedback control cases, while performing on par with the single-agent RL-based adaptive state feedback control, with a much better resilience to partial failures.

Adaptive Tuning of Parameterized Traffic Controllers via Multi-Agent Reinforcement Learning

TL;DR

The paper addresses congestion in transportation networks by integrating reinforcement learning with parameterized state-feedback controllers. It proposes a multi-agent RL framework where each agent tunes its associated controller's parameters at a slower cadence, preserving fast reactive control. The approach offers decentralization, improved training efficiency, and resilience to partial failures, and is evaluated on a multi-class METANET freeway network showing favorable performance against fixed-parameter controls and comparable results to a centralized RL baseline. The results highlight robustness to disturbances and the benefit of continuous-action parameter tuning over discrete parameter sets.

Abstract

Effective traffic control is essential for mitigating congestion in transportation networks. Conventional traffic management strategies, including route guidance, ramp metering, and traffic signal control, often rely on state feedback controllers, used for their simplicity and reactivity; however, they lack the adaptability required to cope with complex and time-varying traffic dynamics. This paper proposes a multi-agent reinforcement learning framework in which each agent adaptively tunes the parameters of a state feedback traffic controller, combining the reactivity of state feedback controllers with the adaptability of reinforcement learning. By tuning parameters at a lower frequency rather than directly determining control actions at a high frequency, the reinforcement learning agents achieve improved training efficiency while maintaining adaptability to varying traffic conditions. The multi-agent structure further enhances system robustness, as local controllers can operate independently in the event of partial failures. The proposed framework is evaluated on a simulated multi-class transportation network under varying traffic conditions. Results show that the proposed multi-agent framework outperforms the no control and fixed-parameter state feedback control cases, while performing on par with the single-agent RL-based adaptive state feedback control, with a much better resilience to partial failures.

Paper Structure

This paper contains 14 sections, 8 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement Learning for Traffic Control
  4. Adaptive Parameterized Traffic Control
  5. Proposed Multi-Agent RL-based Adaptive Parameterized Control
  6. Parameterized Traffic Controllers
  7. Multi-Agent RL Framework for Adaptive Parameter Tuning
  8. Case Study
  9. Transportation Network Setting
  10. Parameterized State Feedback Traffic Controllers
  11. RL Frameworks
  12. Simulation Scenarios
  13. Simulation Results and Discussion
  14. Conclusions

Figures (4)

  • Figure 1: Block diagram of the proposed multi-agent RL-based framework for adaptive tuning of parameterized controllers.
  • Figure 2: Layout of the case study freeway network.
  • Figure 3: The vehicle demands for vehicle classes $\mathrm{c}_1$ and $\mathrm{c}_2$ for the case study.
  • Figure 4: (a) Mean and standard deviation of episode rewards of the RL frameworks, smoothed using a moving average filter of size 40 to better illustrate learning progress. (b) Mean TTS values of the RL frameworks trained with different random seeds.