Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Traffic Control in Complex Dynamic Roadblocks: A Multi-Agent Deep RL Approach

Noor Aboueleneen, Yahuza Bello, Abdullatif Albaseer, Ahmed Refaey Hussein, Mohamed Abdallah, Ekram Hossain

TL;DR

This paper tackles autonomous highway driving under dynamic roadblocks by formulating a decentralized MARL approach using MADDPG and 6G-V2X communications. It defines an MDP where each AV optimizes lane-changing and car-following decisions to maximize the mean harmonic speed $V_h(t)$ under speed, roadblock, and lane-change constraints, and validates the method with SUMO/TraCI simulations. Key findings include robust convergence across scenarios, significant improvements over benchmarks, and reduced Lane-Change frequency, indicating smoother traffic flow and better roadblock avoidance. The work advances real-time, scalable coordination among AVs in ITS, enabling safer and more efficient operations in the presence of dynamic disruptions.

Abstract

Autonomous Vehicles (AVs) represent a transformative advancement in the transportation industry. These vehicles have sophisticated sensors, advanced algorithms, and powerful computing systems that allow them to navigate and operate without direct human intervention. However, AVs' systems still get overwhelmed when they encounter a complex dynamic change in the environment resulting from an accident or a roadblock for maintenance. The advanced features of Sixth Generation (6G) technology are set to offer strong support to AVs, enabling real-time data exchange and management of complex driving maneuvers. This paper proposes a Multi-Agent Reinforcement Learning (MARL) framework to improve AVs' decision-making in dynamic and complex Intelligent Transportation Systems (ITS) utilizing 6G-V2X communication. The primary objective is to enable AVs to avoid roadblocks efficiently by changing lanes while maintaining optimal traffic flow and maximizing the mean harmonic speed. To ensure realistic operations, key constraints such as minimum vehicle speed, roadblock count, and lane change frequency are integrated. We train and test the proposed MARL model with two traffic simulation scenarios using the SUMO and TraCI interface. Through extensive simulations, we demonstrate that the proposed model adapts to various traffic conditions and achieves efficient and robust traffic flow management. The trained model effectively navigates dynamic roadblocks, promoting improved traffic efficiency in AV operations with more than 70% efficiency over other benchmark solutions.

Distributed Traffic Control in Complex Dynamic Roadblocks: A Multi-Agent Deep RL Approach

TL;DR

This paper tackles autonomous highway driving under dynamic roadblocks by formulating a decentralized MARL approach using MADDPG and 6G-V2X communications. It defines an MDP where each AV optimizes lane-changing and car-following decisions to maximize the mean harmonic speed under speed, roadblock, and lane-change constraints, and validates the method with SUMO/TraCI simulations. Key findings include robust convergence across scenarios, significant improvements over benchmarks, and reduced Lane-Change frequency, indicating smoother traffic flow and better roadblock avoidance. The work advances real-time, scalable coordination among AVs in ITS, enabling safer and more efficient operations in the presence of dynamic disruptions.

Abstract

Autonomous Vehicles (AVs) represent a transformative advancement in the transportation industry. These vehicles have sophisticated sensors, advanced algorithms, and powerful computing systems that allow them to navigate and operate without direct human intervention. However, AVs' systems still get overwhelmed when they encounter a complex dynamic change in the environment resulting from an accident or a roadblock for maintenance. The advanced features of Sixth Generation (6G) technology are set to offer strong support to AVs, enabling real-time data exchange and management of complex driving maneuvers. This paper proposes a Multi-Agent Reinforcement Learning (MARL) framework to improve AVs' decision-making in dynamic and complex Intelligent Transportation Systems (ITS) utilizing 6G-V2X communication. The primary objective is to enable AVs to avoid roadblocks efficiently by changing lanes while maintaining optimal traffic flow and maximizing the mean harmonic speed. To ensure realistic operations, key constraints such as minimum vehicle speed, roadblock count, and lane change frequency are integrated. We train and test the proposed MARL model with two traffic simulation scenarios using the SUMO and TraCI interface. Through extensive simulations, we demonstrate that the proposed model adapts to various traffic conditions and achieves efficient and robust traffic flow management. The trained model effectively navigates dynamic roadblocks, promoting improved traffic efficiency in AV operations with more than 70% efficiency over other benchmark solutions.
Paper Structure (18 sections, 19 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 18 sections, 19 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. Multi-Agent Reinforcement Learning
  4. State-of-the-Art Models for Driving Maneuvers
  5. Traffic control models
  6. Car-following and lane-changing models
  7. Integrated car-following and lane-changing models
  8. System Model and Assumptions
  9. Proposed Model and Driving Scenario
  10. Roadblock Control Problem as an MDP
  11. State Definition
  12. Action Definition
  13. Reward Definition
  14. Solution Approach
  15. Performance Evaluation
  16. ...and 3 more sections

Figures (4)

  • Figure 1: MARL-based roadblock control system and highway roadblock scenarios
  • Figure 2: Illustration of the implementation environment using SUMO
  • Figure 3: Convergence of the proposed MADDPG model with comparison to DQN algorithm
  • Figure 6: Harmonic mean speed evaluation for a 3-lane setting