Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A DRL solution to help reduce the cost in waiting time of securing a traffic light for cyclists

Lucas Magnana, Hervé Rivano, Nicolas Chiabaut

TL;DR

This work tackles the challenge of safely isolating cyclists at intersections with dedicated green phases by proposing a 3DQN-based adaptive traffic-light controller. The method learns to sequence and time cyclist-specific greens to minimize total waiting time, using two-channel state representations of lane occupancy and speed and a negative quadratic reward r_t = $-(w_b + w_c)^2$ that penalizes both bike and car queuing. Evaluation on a realistic, day-long simulation driven by real hourly counts from Paris shows the 3DQN approach generally outperforms static and actuated baselines, offering superior performance at moderate traffic levels and robustness to moderate bike-traffic fluctuations; however, performance can degrade when bike demand deviates substantially from training conditions. The work highlights the potential of DRL to enable lightweight, dynamic cyclist-friendly intersections, with practical extensions toward multi-intersection coordination and policy comparisons (e.g., PPO), and provides code for reproducibility.

Abstract

Cyclists prefer to use infrastructure that separates them from motorized traffic. Using a traffic light to segregate car and bike flows, with the addition of bike-specific green phases, is a lightweight and cheap solution that can be deployed dynamically to assess the opportunity of a heavier infrastructure such as a separate bike lane. To compensate for the increased waiting time induced by these new phases, we introduce in this paper a deep reinforcement learning solution that adapts the green phase cycle of a traffic light to the traffic. Vehicle counter data are used to compare the DRL approach with the actuated traffic light control algorithm over whole days. Results show that DRL achieves better minimization of vehicle waiting time at almost all hours. Our DRL approach is also robust to moderate changes in bike traffic. The code of this paper is available at https://github.com/LucasMagnana/A-DRL-solution-to-help-reduce-the-cost-in-waiting-time-of-securing-a-traffic-light-for-cyclists.

A DRL solution to help reduce the cost in waiting time of securing a traffic light for cyclists

TL;DR

This work tackles the challenge of safely isolating cyclists at intersections with dedicated green phases by proposing a 3DQN-based adaptive traffic-light controller. The method learns to sequence and time cyclist-specific greens to minimize total waiting time, using two-channel state representations of lane occupancy and speed and a negative quadratic reward r_t = that penalizes both bike and car queuing. Evaluation on a realistic, day-long simulation driven by real hourly counts from Paris shows the 3DQN approach generally outperforms static and actuated baselines, offering superior performance at moderate traffic levels and robustness to moderate bike-traffic fluctuations; however, performance can degrade when bike demand deviates substantially from training conditions. The work highlights the potential of DRL to enable lightweight, dynamic cyclist-friendly intersections, with practical extensions toward multi-intersection coordination and policy comparisons (e.g., PPO), and provides code for reproducibility.

Abstract

Cyclists prefer to use infrastructure that separates them from motorized traffic. Using a traffic light to segregate car and bike flows, with the addition of bike-specific green phases, is a lightweight and cheap solution that can be deployed dynamically to assess the opportunity of a heavier infrastructure such as a separate bike lane. To compensate for the increased waiting time induced by these new phases, we introduce in this paper a deep reinforcement learning solution that adapts the green phase cycle of a traffic light to the traffic. Vehicle counter data are used to compare the DRL approach with the actuated traffic light control algorithm over whole days. Results show that DRL achieves better minimization of vehicle waiting time at almost all hours. Our DRL approach is also robust to moderate changes in bike traffic. The code of this paper is available at https://github.com/LucasMagnana/A-DRL-solution-to-help-reduce-the-cost-in-waiting-time-of-securing-a-traffic-light-for-cyclists.
Paper Structure (28 sections, 7 equations, 7 figures, 1 table)

This paper contains 28 sections, 7 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Cyclists and traffic lights
  3. DRL for traffic light control
  4. Positioning and contributions
  5. Deep reinforcement learning
  6. Deep Q-Network
  7. Double Deep Q-Network
  8. Dueling Deep Q-Network
  9. Double Dueling Deep Q-Network (3DQN)
  10. 3DQN approach
  11. Environment
  12. States
  13. Actions
  14. Rewards
  15. Reward function
  16. ...and 13 more sections

Figures (7)

  • Figure 1: Diagram showing the construction of the position matrix from an image of an intersection.
  • Figure 2: Diagram showing the structure of the Q-network.
  • Figure 3: Screenshot of the environment simulated by SUMO.
  • Figure 4: Average number of vehicles counted per hour in both directions on boulevard Montparnasse on June 20, 2023.
  • Figure 5: Hourly number of vehicles and mean waiting time for a simulation of one day.
  • ...and 2 more figures