Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning

Bibek Poudel, Weizi Li, Kevin Heaslip

TL;DR

This work introduces a reinforcement learning based RV that employs a congestion stage classifier to optimize the safety, efficiency, and stability of mixed traffic.

Abstract

Human-driven vehicles (HVs) amplify naturally occurring perturbations in traffic, leading to congestion--a major contributor to increased fuel consumption, higher collision risks, and reduced road capacity utilization. While previous research demonstrates that Robot Vehicles (RVs) can be leveraged to mitigate these issues, most such studies rely on simulations with simplistic models of human car-following behaviors. In this work, we analyze real-world driving trajectories and extract a wide range of acceleration profiles. We then incorporates these profiles into simulations for training RVs to mitigate congestion. We evaluate the safety, efficiency, and stability of mixed traffic via comprehensive experiments conducted in two mixed traffic environments (Ring and Bottleneck) at various traffic densities, configurations, and RV penetration rates. The results show that under real-world perturbations, prior RV controllers experience performance degradation on all three objectives (sometimes even lower than 100% HVs). To address this, we introduce a reinforcement learning based RV that employs a congestion stage classifier to optimize the safety, efficiency, and stability of mixed traffic. Our RVs demonstrate significant improvements: safety by up to 66%, efficiency by up to 54%, and stability by up to 97%.

EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning

TL;DR

This work introduces a reinforcement learning based RV that employs a congestion stage classifier to optimize the safety, efficiency, and stability of mixed traffic.

Abstract

Human-driven vehicles (HVs) amplify naturally occurring perturbations in traffic, leading to congestion--a major contributor to increased fuel consumption, higher collision risks, and reduced road capacity utilization. While previous research demonstrates that Robot Vehicles (RVs) can be leveraged to mitigate these issues, most such studies rely on simulations with simplistic models of human car-following behaviors. In this work, we analyze real-world driving trajectories and extract a wide range of acceleration profiles. We then incorporates these profiles into simulations for training RVs to mitigate congestion. We evaluate the safety, efficiency, and stability of mixed traffic via comprehensive experiments conducted in two mixed traffic environments (Ring and Bottleneck) at various traffic densities, configurations, and RV penetration rates. The results show that under real-world perturbations, prior RV controllers experience performance degradation on all three objectives (sometimes even lower than 100% HVs). To address this, we introduce a reinforcement learning based RV that employs a congestion stage classifier to optimize the safety, efficiency, and stability of mixed traffic. Our RVs demonstrate significant improvements: safety by up to 66%, efficiency by up to 54%, and stability by up to 97%.
Paper Structure (8 sections, 9 equations, 5 figures, 3 tables)

This paper contains 8 sections, 9 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Intelligent Driver Model (IDM)
  5. Sampling Real-world Perturbations
  6. Reinforcement Learning with Congestion Stage Classifier
  7. Experiments
  8. Conclusion and Discussion

Figures (5)

  • Figure 1: TOP: The congestion stage classifier takes the position and velocity of the leader HVs of RV in the sensing zone to predict the congestion stage $10$ timesteps into the future, enabling pro-active responses of the RV. MIDDLE: Our RVs deployed at various penetration rates in Ring. When penetration rates $> 5\%$, RVs are arranged as a platoon. BOTTOM: Our RVs deployed at $40$% penetration rate in Bottleneck (truncated version shown).
  • Figure 2: Progressive amplification of longitudinal perturbation in the absence of RVs, and the strategies adopted by various RVs to attenuate such perturbations and stabilize traffic ($6$ HVs following the RVs are shown). Wave attenuation characteristics of various RVs at $81~veh/km$ are provided. The HV immediately in front of RV(s) produces a shock by applying a standard velocity perturbation of $3~m/s$. With IDM (100% HVs), perturbation amplifies over time, whereas RVs dampen the perturbation over time.
  • Figure 3: Instantaneous accelerations observed during car-following behaviors at densities $[70, 150]~veh/km$. TOP: Real-world data from the I-$24$ MOTION dataset reveals a distribution having long tails extending to $[-3, 3]~m/s^2$. BOTTOM: IDM (in simulation) produces accelerations mostly within $[-0.5, 0.5]~m/s^2$, indicating much 'timid' driving behaviors than the real world.
  • Figure 4: The results of applying K-means clustering with t-SNE on a subset of CSC training data. LEFT: In Ring, the clusters are spread out, suggesting that the data is easily classifiable. RIGHT: In Bottleneck, overlapping clusters indicate that more complex interactions exist among the congestion stages, possibly due to the presence of zipper lanes causing vehicles abruptly merge.
  • Figure 5: Confusion matrix of a trained CSC in Ring (LEFT) and Bottleneck (RIGHT) on the validation set.