Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cooperative Cruising: Reinforcement Learning-Based Time-Headway Control for Increased Traffic Efficiency

Yaron Veksler, Sharon Hornstein, Han Wang, Maria Laura Delle Monache, Daniel Urieli

TL;DR

This paper tackles highway congestion in realistic multi-lane settings by proposing a centralized reinforcement-learning-based controller that emits time-headway commands to CAVs near bottlenecks, integrated with safety-certified ACC and low-bandwidth V2I. The method is trained in large-scale SUMO simulations of a 2 km I-24 highway segment, using an MDP formulation with states over 21 road segments, actions as headway vectors, and a time-delay reward tied to free-flow performance. Empirical results show that the RL controller yields up to 13% average speed improvements in single-lane scenarios and up to 7% in multi-lane scenarios, outperforming a fixed-headway baseline across varying CAV penetration. The work emphasizes practicality and safety, offering deployable gains by targeting localized bottlenecks, and provides a novel average-speed metric for open-road simulations to better reflect real-world traffic dynamics.

Abstract

The proliferation of connected automated vehicles represents an unprecedented opportunity for improving driving efficiency and alleviating traffic congestion. However, existing research fails to address realistic multi-lane highway scenarios without assuming connectivity, perception, and control capabilities that are typically unavailable in current vehicles. This paper proposes a novel AI system that is the first to improve highway traffic efficiency compared with human-like traffic in realistic, simulated multi-lane scenarios, while relying on existing connectivity, perception, and control capabilities. At the core of our approach is a reinforcement learning based controller that dynamically communicates time-headways to automated vehicles near bottlenecks based on real-time traffic conditions. These desired time-headways are then used by adaptive cruise control (ACC) systems to adjust their following distance. By (i) integrating existing traffic estimation technology and low-bandwidth vehicle-to-infrastructure connectivity, (ii) leveraging safety-certified ACC systems, and (iii) targeting localized bottleneck challenges that can be addressed independently in different locations, we propose a potentially practical, safe, and scalable system that can positively impact numerous road users.

Cooperative Cruising: Reinforcement Learning-Based Time-Headway Control for Increased Traffic Efficiency

TL;DR

This paper tackles highway congestion in realistic multi-lane settings by proposing a centralized reinforcement-learning-based controller that emits time-headway commands to CAVs near bottlenecks, integrated with safety-certified ACC and low-bandwidth V2I. The method is trained in large-scale SUMO simulations of a 2 km I-24 highway segment, using an MDP formulation with states over 21 road segments, actions as headway vectors, and a time-delay reward tied to free-flow performance. Empirical results show that the RL controller yields up to 13% average speed improvements in single-lane scenarios and up to 7% in multi-lane scenarios, outperforming a fixed-headway baseline across varying CAV penetration. The work emphasizes practicality and safety, offering deployable gains by targeting localized bottlenecks, and provides a novel average-speed metric for open-road simulations to better reflect real-world traffic dynamics.

Abstract

The proliferation of connected automated vehicles represents an unprecedented opportunity for improving driving efficiency and alleviating traffic congestion. However, existing research fails to address realistic multi-lane highway scenarios without assuming connectivity, perception, and control capabilities that are typically unavailable in current vehicles. This paper proposes a novel AI system that is the first to improve highway traffic efficiency compared with human-like traffic in realistic, simulated multi-lane scenarios, while relying on existing connectivity, perception, and control capabilities. At the core of our approach is a reinforcement learning based controller that dynamically communicates time-headways to automated vehicles near bottlenecks based on real-time traffic conditions. These desired time-headways are then used by adaptive cruise control (ACC) systems to adjust their following distance. By (i) integrating existing traffic estimation technology and low-bandwidth vehicle-to-infrastructure connectivity, (ii) leveraging safety-certified ACC systems, and (iii) targeting localized bottleneck challenges that can be addressed independently in different locations, we propose a potentially practical, safe, and scalable system that can positively impact numerous road users.

Paper Structure

This paper contains 34 sections, 13 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Domain Description
  4. Problem Description
  5. Simulation Platform & Scenario Parameters
  6. Methodology
  7. Congestion Metrics
  8. Centralized Time-Headway Control
  9. Practical Design Choices
  10. Fixed-Valued Time-Headway Control
  11. RL-based Time-Headway Control
  12. MDP
  13. Modeling Traffic Control as an MDP
  14. States
  15. Actions
  16. ...and 19 more sections

Figures (3)

  • Figure 1: Centralized Time-headway control for multi-lane highway congestion reduction: (a) The analyzed scenario. An RL-based controller sends time-headway commands to CAVs near bottlenecks, based on measured traffic speed and density. (b) Lane-changing behavior simulation results. Aggressive lane-changing behavior significantly impacts traffic dynamics.
  • Figure 2: Time-headway control motivation: (a) Aggressive lane-change may cause excessive speed and throughput decrease (top). Preemptively increasing headway can reduce the negative effect (bottom). (b) Analysis of constant speed-limit command dynamics. Low downstream density is maintained for a limited duration (blue). (c) Simulation results for constant speed and time-headway signals. Constant time-headway signals maintain lower downstream density for arbitrary duration.
  • Figure 3: Time-headway control performance: Simulation performance for (a) single-lane, and (b) multi-lane scenarios. Performance of our safe RL-based controller and fixed-valued time-headway control baseline is measured relative to simulated human-driven traffic (dashed black line). Error bars show 95% confidence intervals for mean performance values, each derived from 30 simulations.