Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Emergent Cooperative Driving Strategies for Stop-and-Go Wave Mitigation via Multi-Agent Reinforcement Learning

Raphael Korbmacher, Daniel Straub, Antoine Tordeux, Claudia Totzeck

TL;DR

It is demonstrated that adapting this cooperative strategy to classical car-following models, such as the Intelligent Driver Model (IDM), yields improved stability and traffic efficiency, and within a parametrised linear framework, it can optimise system performance under stability constraints.

Abstract

Stop-and-go waves in traffic flow pose a persistent challenge, compromising safety, efficiency, and environmental sustainability. This paper introduces a novel mitigation strategy discovered through training multi-agent deep reinforcement learning (DRL) agents in a simulated ring-road environment. The agents autonomously develop a cooperative driving policy, where most vehicles maintain minimal headways to maximize throughput, while a single "buffer" vehicle adopts a larger headway to absorb perturbations and prevent wave propagation. This strategy enhances stability without sacrificing overall flow. We further demonstrate that adapting this cooperative strategy to classical car-following models, such as the Intelligent Driver Model (IDM), yields improved stability and traffic efficiency. Furthermore, we show within a parametrised linear framework, that the cooperative strategy can optimise system performance under stability constraints. Our findings offer promising insights for future autonomous vehicle systems and highway management.

Emergent Cooperative Driving Strategies for Stop-and-Go Wave Mitigation via Multi-Agent Reinforcement Learning

TL;DR

It is demonstrated that adapting this cooperative strategy to classical car-following models, such as the Intelligent Driver Model (IDM), yields improved stability and traffic efficiency, and within a parametrised linear framework, it can optimise system performance under stability constraints.

Abstract

Stop-and-go waves in traffic flow pose a persistent challenge, compromising safety, efficiency, and environmental sustainability. This paper introduces a novel mitigation strategy discovered through training multi-agent deep reinforcement learning (DRL) agents in a simulated ring-road environment. The agents autonomously develop a cooperative driving policy, where most vehicles maintain minimal headways to maximize throughput, while a single "buffer" vehicle adopts a larger headway to absorb perturbations and prevent wave propagation. This strategy enhances stability without sacrificing overall flow. We further demonstrate that adapting this cooperative strategy to classical car-following models, such as the Intelligent Driver Model (IDM), yields improved stability and traffic efficiency. Furthermore, we show within a parametrised linear framework, that the cooperative strategy can optimise system performance under stability constraints. Our findings offer promising insights for future autonomous vehicle systems and highway management.

Paper Structure

This paper contains 13 sections, 34 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Deep Reinforcement Learning
  3. Experimental setup
  4. Training
  5. Emergent cooperative strategy
  6. Model-based Validation of the Cooperative Strategy
  7. Intelligent Driver car-following model
  8. Simulation results
  9. Model-based strategy optimization
  10. Optimal stability control problem
  11. Non-cooperative optimal solution
  12. Cooperative optimal solution
  13. Conclusion and Outlook

Figures (5)

  • Figure 2: Typical observation of the multi-agent DRL. The headway distances of almost all vehicles are close to the safety-critical time gap. Only one vehicle (cyan) maintains a very large headway.
  • Figure 3: Simulation results using identical IDM parameter settings, without (left panels) and with (right panels) the buffering cooperative strategy. Both scenarios start from the same initial traffic jam condition. Top row: vehicle trajectories; bottom row: corresponding speed profiles. Without the strategy (left) a stop-and-go wave emerges and propagates upstream. With the buffering cooperative strategy (right) the traffic flow converges to homogeneous laminar flow. The trajectories can be computed and visualized online; see SimBufferingStrategy.
  • Figure : (a) Gunter et al. (2018) Experiment
  • Figure : (a) Gunter et al. (2018) Experiment
  • Figure : (b) Stable Car-Following Model