Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Z-Merge: Multi-Agent Reinforcement Learning for On-Ramp Merging with Zone-Specific V2X Traffic Information

Yassine Ibork, Myounggyu Won, Lokesh Das

TL;DR

Z-Merge tackles on-ramp merging in mixed traffic by introducing a zone-based MARL framework that leverages RSU-provided zone-specific traffic information. It formulates merging as a MA-POMDP with a hybrid discrete-continuous action space and trains via a Centralized Training with Decentralized Execution using a Double PDQN approach. The method integrates zone-level global data with local sensing to coordinate lane-changing and gap adjustments, yielding significant gains in efficiency, safety, driving comfort, and queue management across diverse traffic conditions, with real-time inference performance. The work demonstrates practical impact for scalable, V2X-enabled autonomous-vehicle coordination in complex merging scenarios.

Abstract

Ramp merging is a critical and challenging task for autonomous vehicles (AVs), particularly in mixed traffic environments with human-driven vehicles (HVs). Existing approaches typically rely on either lane-changing or inter-vehicle gap creation strategies based solely on local or neighboring information, often leading to suboptimal performance in terms of safety and traffic efficiency. In this paper, we present a V2X (vehicle-to-everything communication)-assisted Multiagent Reinforcement Learning (MARL) framework for on-ramp merging that effectively coordinates the complex interplay between lane-changing and inter-vehicle gap adaptation strategies by utilizing zone-specific global information available from a roadside unit (RSU). The merging control problem is formulated as a Multiagent Partially Observable Markov Decision Process (MA-POMDP), where agents leverage both local and global observations through V2X communication. To support both discrete and continuous control decisions, we design a hybrid action space and adopt a parameterized deep Q-learning approach. Extensive simulations, integrating the SUMO traffic simulator and the MOSAIC V2X simulator, demonstrate that our framework significantly improves merging success rate, traffic efficiency, and road safety across diverse traffic scenarios.

Z-Merge: Multi-Agent Reinforcement Learning for On-Ramp Merging with Zone-Specific V2X Traffic Information

TL;DR

Z-Merge tackles on-ramp merging in mixed traffic by introducing a zone-based MARL framework that leverages RSU-provided zone-specific traffic information. It formulates merging as a MA-POMDP with a hybrid discrete-continuous action space and trains via a Centralized Training with Decentralized Execution using a Double PDQN approach. The method integrates zone-level global data with local sensing to coordinate lane-changing and gap adjustments, yielding significant gains in efficiency, safety, driving comfort, and queue management across diverse traffic conditions, with real-time inference performance. The work demonstrates practical impact for scalable, V2X-enabled autonomous-vehicle coordination in complex merging scenarios.

Abstract

Ramp merging is a critical and challenging task for autonomous vehicles (AVs), particularly in mixed traffic environments with human-driven vehicles (HVs). Existing approaches typically rely on either lane-changing or inter-vehicle gap creation strategies based solely on local or neighboring information, often leading to suboptimal performance in terms of safety and traffic efficiency. In this paper, we present a V2X (vehicle-to-everything communication)-assisted Multiagent Reinforcement Learning (MARL) framework for on-ramp merging that effectively coordinates the complex interplay between lane-changing and inter-vehicle gap adaptation strategies by utilizing zone-specific global information available from a roadside unit (RSU). The merging control problem is formulated as a Multiagent Partially Observable Markov Decision Process (MA-POMDP), where agents leverage both local and global observations through V2X communication. To support both discrete and continuous control decisions, we design a hybrid action space and adopt a parameterized deep Q-learning approach. Extensive simulations, integrating the SUMO traffic simulator and the MOSAIC V2X simulator, demonstrate that our framework significantly improves merging success rate, traffic efficiency, and road safety across diverse traffic scenarios.

Paper Structure

This paper contains 18 sections, 16 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Design of Z-Merge
  4. Overview
  5. Problem Formulation
  6. State Space:
  7. Action Space:
  8. Reward Function:
  9. Multi-Agent Parameterized Double DQN
  10. Simulation Results
  11. Simulation Setup
  12. Traffic Efficiency
  13. Traffic Safety
  14. Driving Comfort
  15. Queue Length
  16. ...and 3 more sections

Figures (6)

  • Figure 1: An operational overview of Z-Merge.
  • Figure 2: The average speed changes over time.
  • Figure 3: Spatio-temporal representation of the average speed for Z-Merge in pre-merging and merging zones.
  • Figure 4: Acceleration trajectories for vehicles at three different time instances. The top row presents the acceleration profile of a merging vehicle, whereas the bottom row presents that of a mainline vehicle. Z-Merge, Baseline-1, Baseline-2, Baseline-3.
  • Figure 5: Avg. queue length.
  • ...and 1 more figures