Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neighbor-Aware Reinforcement Learning for Mixed Traffic Optimization in Large-scale Networks

Iftekharul Islam, Weizi Li

TL;DR

This work tackles traffic management in large urban networks with mixed human-driven and robot vehicles. It introduces a neighbor-aware reinforcement learning framework based on a network-level POMDP and Rainbow DQN, designed to coordinate RVs across interconnected intersections while balancing RV distribution. By incorporating local efficiency, conflict avoidance, and neighbor distribution components into the reward, the method achieves substantial reductions in average waiting times on a real-world 17-intersection Colorado Springs network, outperforming both single-intersection RL and traditional signal controls (up to 79.8% faster than signalized signals). The results demonstrate the practicality and effectiveness of learning-based, network-wide coordination for mixed-autonomy urban traffic and establish a foundation for deploying such solutions in real cities.

Abstract

Managing mixed traffic comprising human-driven and robot vehicles (RVs) across large-scale networks presents unique challenges beyond single-intersection control. This paper proposes a reinforcement learning framework for coordinating mixed traffic across multiple interconnected intersections. Our key contribution is a neighbor-aware reward mechanism that enables RVs to maintain balanced distribution across the network while optimizing local intersection efficiency. We evaluate our approach using a real-world network, demonstrating its effectiveness in managing realistic traffic patterns. Results show that our method reduces average waiting times by 39.2% compared to the state-of-the-art single-intersection control policy and 79.8% compared to traditional traffic signals. The framework's ability to coordinate traffic across multiple intersections while maintaining balanced RV distribution provides a foundation for deploying learning-based solutions in urban traffic systems.

Neighbor-Aware Reinforcement Learning for Mixed Traffic Optimization in Large-scale Networks

TL;DR

This work tackles traffic management in large urban networks with mixed human-driven and robot vehicles. It introduces a neighbor-aware reinforcement learning framework based on a network-level POMDP and Rainbow DQN, designed to coordinate RVs across interconnected intersections while balancing RV distribution. By incorporating local efficiency, conflict avoidance, and neighbor distribution components into the reward, the method achieves substantial reductions in average waiting times on a real-world 17-intersection Colorado Springs network, outperforming both single-intersection RL and traditional signal controls (up to 79.8% faster than signalized signals). The results demonstrate the practicality and effectiveness of learning-based, network-wide coordination for mixed-autonomy urban traffic and establish a foundation for deploying such solutions in real cities.

Abstract

Managing mixed traffic comprising human-driven and robot vehicles (RVs) across large-scale networks presents unique challenges beyond single-intersection control. This paper proposes a reinforcement learning framework for coordinating mixed traffic across multiple interconnected intersections. Our key contribution is a neighbor-aware reward mechanism that enables RVs to maintain balanced distribution across the network while optimizing local intersection efficiency. We evaluate our approach using a real-world network, demonstrating its effectiveness in managing realistic traffic patterns. Results show that our method reduces average waiting times by 39.2% compared to the state-of-the-art single-intersection control policy and 79.8% compared to traditional traffic signals. The framework's ability to coordinate traffic across multiple intersections while maintaining balanced RV distribution provides a foundation for deploying learning-based solutions in urban traffic systems.

Paper Structure

This paper contains 11 sections, 2 equations, 2 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. METHODOLOGY
  4. RL-based Network-Level Control Framework
  5. Network-Aware State Representation
  6. Neighbor-Aware Reward Design
  7. Large-Scale Network Architecture
  8. Experiments and Results
  9. Experimental Setup
  10. Overall Performance
  11. CONCLUSION AND FUTURE WORK

Figures (2)

  • Figure 1: Overview of the neighbor-aware reinforcement learning framework for large-scale mixed traffic control. The system operates in a continuous loop where each RV observes the network state (including queue length, waiting time, occupancy map, and downstream conditions), processes this information through a Rainbow DQN policy network (three hidden layers with 512 units each), and selects either Stop or Go actions. The reward function combines three components: local efficiency ($R_{local}$), conflict avoidance ($R_{conflict}$), and neighbor-aware distribution ($R_{neighbor}$). This feedback loop enables the system to learn policies that balance local intersection efficiency with network-wide RV distribution.
  • Figure 2: Network representation of the study area in Colorado Springs, CO, USA. (a) Google Maps visualization showing typical traffic conditions during Monday evening rush hour (4:55 PM). (b) Simplified network after removing secondary roads.