Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Macroscopic Characteristics of Mixed Traffic Flow with Deep Reinforcement Learning Based Automated and Human-Driven Vehicles

Pankaj Kumar, Pranamesh Chakraborty, Subrahmanya Swamy Peruru

Abstract

Automated Vehicle (AV) control in mixed traffic, where AVs coexist with human-driven vehicles, poses significant challenges in balancing safety, efficiency, comfort, fuel efficiency, and compliance with traffic rules while capturing heterogeneous driver behavior. Traditional car-following models, such as the Intelligent Driver Model (IDM), often struggle to generalize across diverse traffic scenarios and typically do not account for fuel efficiency, motivating the use of learning-based approaches. Although Deep Reinforcement Learning (DRL) has shown strong microscopic performance in car-following conditions, its macroscopic traffic flow characteristics remain underexplored. This study focuses on analyzing the macroscopic traffic flow characteristics and fuel efficiency of DRL-based models in mixed traffic. A Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is implemented for AVs' control and trained using the NGSIM highway dataset, enabling realistic interaction with human-driven vehicles. Traffic performance is evaluated using the Fundamental Diagram (FD) under varying driver heterogeneity, heterogeneous time-gap penetration levels, and different shares of RL-controlled vehicles. A macroscopic level comparison of fuel efficiency between the RL-based AV model and the IDM is also conducted. Results show that traffic performance is sensitive to the distribution of safe time gaps and the proportion of RL vehicles. Transitioning from fully human-driven to fully RL-controlled traffic can increase road capacity by approximately 7.52%. Further, RL-based AVs also improve average fuel efficiency by about 28.98% at higher speeds (above 50 km/h), and by 1.86% at lower speeds (below 50 km/h) compared to the IDM. Overall, the DRL framework enhances traffic capacity and fuel efficiency without compromising safety.

Macroscopic Characteristics of Mixed Traffic Flow with Deep Reinforcement Learning Based Automated and Human-Driven Vehicles

Abstract

Automated Vehicle (AV) control in mixed traffic, where AVs coexist with human-driven vehicles, poses significant challenges in balancing safety, efficiency, comfort, fuel efficiency, and compliance with traffic rules while capturing heterogeneous driver behavior. Traditional car-following models, such as the Intelligent Driver Model (IDM), often struggle to generalize across diverse traffic scenarios and typically do not account for fuel efficiency, motivating the use of learning-based approaches. Although Deep Reinforcement Learning (DRL) has shown strong microscopic performance in car-following conditions, its macroscopic traffic flow characteristics remain underexplored. This study focuses on analyzing the macroscopic traffic flow characteristics and fuel efficiency of DRL-based models in mixed traffic. A Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is implemented for AVs' control and trained using the NGSIM highway dataset, enabling realistic interaction with human-driven vehicles. Traffic performance is evaluated using the Fundamental Diagram (FD) under varying driver heterogeneity, heterogeneous time-gap penetration levels, and different shares of RL-controlled vehicles. A macroscopic level comparison of fuel efficiency between the RL-based AV model and the IDM is also conducted. Results show that traffic performance is sensitive to the distribution of safe time gaps and the proportion of RL vehicles. Transitioning from fully human-driven to fully RL-controlled traffic can increase road capacity by approximately 7.52%. Further, RL-based AVs also improve average fuel efficiency by about 28.98% at higher speeds (above 50 km/h), and by 1.86% at lower speeds (below 50 km/h) compared to the IDM. Overall, the DRL framework enhances traffic capacity and fuel efficiency without compromising safety.

Paper Structure

This paper contains 14 sections, 16 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Design Methodology
  3. Action space and state dynamics
  4. Reward formulation
  5. Reinforcement learning (RL) algorithm description
  6. Estimation of macroscopic traffic variables
  7. Data description and environment setup
  8. Results and Interpretation
  9. Training result
  10. Fundamental diagram under driver heterogeneity
  11. Fundamental diagram under varying penetration rates of driver heterogeneity
  12. Fundamental diagram of mixed traffic flow under varying penetration rates
  13. Comparative fuel efficiency analysis
  14. Conclusion

Figures (5)

  • Figure 1: General schematic representation of (a) time-space region to measure traffic flow characteristics, and (b) parallelogram region with trajectory points inside
  • Figure 2: Example leader vehicle trajectories produced by the Ornstein–Uhlenbeck process
  • Figure 3: Normalized rolling rewards of the TD3 agent during training
  • Figure 4: Visualisation of traffic flow analysis: (a) time–space trajectory data with generated parallelogram, (b) magnified section of region A, and (c) empirical fundamental diagrams under varying levels of driver heterogeneity
  • Figure 5: Fuel efficiency performance of RL and IDM models: (a) average fuel efficiency, and (b) % of time with positive acceleration across speed ranges