Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Agent Car Parking using Reinforcement Learning

Omar Tanner

TL;DR

The paper investigates safe, efficient multi-agent autonomous parking using reinforcement learning within a configurable Unity-based environment. It formulates parking as MDPs with independent PPO learners, enabling both fixed and dynamic goals, local pose encoding, and inter-agent communication to study competition and collaboration. Empirical results show high parking success (up to 99%) with multiple agents, improved performance when goals are shared, and nuanced behaviors around collaboration via giving-way contexts; dynamic goals foster richer interactions and emergent strategies. The work provides scalable benchmarks, methodological tools, and insights for MARL in fleet management, with clear avenues for enhancing realism, safety, and infrastructure for real-world deployment.

Abstract

As the industry of autonomous driving grows, so does the potential interaction of groups of autonomous cars. Combined with the advancement of Artificial Intelligence and simulation, such groups can be simulated, and safety-critical models can be learned controlling the cars within. This study applies reinforcement learning to the problem of multi-agent car parking, where groups of cars aim to efficiently park themselves, while remaining safe and rational. Utilising robust tools and machine learning frameworks, we design and implement a flexible car parking environment in the form of a Markov decision process with independent learners, exploiting multi-agent communication. We implement a suite of tools to perform experiments at scale, obtaining models parking up to 7 cars with over a 98.1% success rate, significantly beating existing single-agent models. We also obtain several results relating to competitive and collaborative behaviours exhibited by the cars in our environment, with varying densities and levels of communication. Notably, we discover a form of collaboration that cannot arise without competition, and a 'leaky' form of collaboration whereby agents collaborate without sufficient state. Such work has numerous potential applications in the autonomous driving and fleet management industries, and provides several useful techniques and benchmarks for the application of reinforcement learning to multi-agent car parking.

Multi-Agent Car Parking using Reinforcement Learning

TL;DR

The paper investigates safe, efficient multi-agent autonomous parking using reinforcement learning within a configurable Unity-based environment. It formulates parking as MDPs with independent PPO learners, enabling both fixed and dynamic goals, local pose encoding, and inter-agent communication to study competition and collaboration. Empirical results show high parking success (up to 99%) with multiple agents, improved performance when goals are shared, and nuanced behaviors around collaboration via giving-way contexts; dynamic goals foster richer interactions and emergent strategies. The work provides scalable benchmarks, methodological tools, and insights for MARL in fleet management, with clear avenues for enhancing realism, safety, and infrastructure for real-world deployment.

Abstract

As the industry of autonomous driving grows, so does the potential interaction of groups of autonomous cars. Combined with the advancement of Artificial Intelligence and simulation, such groups can be simulated, and safety-critical models can be learned controlling the cars within. This study applies reinforcement learning to the problem of multi-agent car parking, where groups of cars aim to efficiently park themselves, while remaining safe and rational. Utilising robust tools and machine learning frameworks, we design and implement a flexible car parking environment in the form of a Markov decision process with independent learners, exploiting multi-agent communication. We implement a suite of tools to perform experiments at scale, obtaining models parking up to 7 cars with over a 98.1% success rate, significantly beating existing single-agent models. We also obtain several results relating to competitive and collaborative behaviours exhibited by the cars in our environment, with varying densities and levels of communication. Notably, we discover a form of collaboration that cannot arise without competition, and a 'leaky' form of collaboration whereby agents collaborate without sufficient state. Such work has numerous potential applications in the autonomous driving and fleet management industries, and provides several useful techniques and benchmarks for the application of reinforcement learning to multi-agent car parking.
Paper Structure (115 sections, 2 theorems, 17 equations, 36 figures, 12 tables)

This paper contains 115 sections, 2 theorems, 17 equations, 36 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Single-Agent RL
  4. Multi-Agent RL
  5. Implementation Methods
  6. Objectives
  7. Background
  8. Environment
  9. Reinforcement Learning
  10. Q-Learning
  11. Information in MDPs
  12. Deep Reinforcement Learning
  13. Deep Q-Learning
  14. PPO
  15. Multi-Agent Reinforcement Learning
  16. ...and 100 more sections

Key Result

Theorem 1

Let $\langle\mathbb{S}, \mathbb{A}, \mathbb{P}, \mathbb{R}, \mathbb{O}, \Omega \rangle$ be a fully deterministic deterministic POMDP with full information extending the deterministic MDP $\langle\mathbb{S}, \mathbb{A}, \mathbb{P}, \mathbb{R} \rangle$, and policies $\pi : \mathbb{S} \to \mathbb{A}$ a

Figures (36)

  • Figure 1: The actor-critic architecture, depicted by rl-book.
  • Figure 2: Our high-level methodology.
  • Figure 3: Environment schematics.
  • Figure 4: Pose information for nearby objects.
  • Figure 5: Discretised parking space and car dimensions.
  • ...and 31 more figures

Theorems & Definitions (6)

  • Definition 1: Full Information Reduced MDP
  • Theorem 1: Optimal Policy Learning in Full Information Reduced MDPs
  • proof
  • Theorem 2: Deadlock Free Evolution
  • proof
  • proof