Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Distributed Approach to Autonomous Intersection Management via Multi-Agent Reinforcement Learning

Matteo Cederle, Marco Fabris, Gian Antonio Susto

TL;DR

It is shown that by leveraging the 3D surround view technology for advanced assistance systems, autonomous vehicles can accurately navigate intersection scenarios without needing any centralised controller.

Abstract

Autonomous intersection management (AIM) poses significant challenges due to the intricate nature of real-world traffic scenarios and the need for a highly expensive centralised server in charge of simultaneously controlling all the vehicles. This study addresses such issues by proposing a novel distributed approach to AIM utilizing multi-agent reinforcement learning (MARL). We show that by leveraging the 3D surround view technology for advanced assistance systems, autonomous vehicles can accurately navigate intersection scenarios without needing any centralised controller. The contributions of this paper thus include a MARL-based algorithm for the autonomous management of a 4-way intersection and also the introduction of a new strategy called prioritised scenario replay for improved training efficacy. We validate our approach as an innovative alternative to conventional centralised AIM techniques, ensuring the full reproducibility of our results. Specifically, experiments conducted in virtual environments using the SMARTS platform highlight its superiority over benchmarks across various metrics.

A Distributed Approach to Autonomous Intersection Management via Multi-Agent Reinforcement Learning

TL;DR

It is shown that by leveraging the 3D surround view technology for advanced assistance systems, autonomous vehicles can accurately navigate intersection scenarios without needing any centralised controller.

Abstract

Autonomous intersection management (AIM) poses significant challenges due to the intricate nature of real-world traffic scenarios and the need for a highly expensive centralised server in charge of simultaneously controlling all the vehicles. This study addresses such issues by proposing a novel distributed approach to AIM utilizing multi-agent reinforcement learning (MARL). We show that by leveraging the 3D surround view technology for advanced assistance systems, autonomous vehicles can accurately navigate intersection scenarios without needing any centralised controller. The contributions of this paper thus include a MARL-based algorithm for the autonomous management of a 4-way intersection and also the introduction of a new strategy called prioritised scenario replay for improved training efficacy. We validate our approach as an innovative alternative to conventional centralised AIM techniques, ensuring the full reproducibility of our results. Specifically, experiments conducted in virtual environments using the SMARTS platform highlight its superiority over benchmarks across various metrics.
Paper Structure (13 sections, 11 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 13 sections, 11 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Theoretical background
  3. Basic notions of reinforcement learning
  4. Q-Learning and Deep Q-Networks
  5. Multi-agent reinforcement learning
  6. Multi-Agent Decentralised Dueling Double Deep Q-Networks with Prioritized Scenario replay
  7. System modelling and design
  8. Learning Strategy
  9. The prioritised scenario replay strategy
  10. Experiments on virtual environments
  11. Baselines
  12. Results
  13. Conclusions and future directions

Figures (2)

  • Figure 1: Visual representation of the 4-way 1-lane intersection environment considered for the experiments (image created through SUMO simulator SUMO2018).
  • Figure 2: Comparison between the performance metrics of our method (MAD4QN-PS) and the baselines introduced in Section \ref{['subsec:baselines']}.