Work Smarter Not Harder: Simple Imitation Learning with CS-PIBT Outperforms Large Scale Imitation Learning for MAPF

TL;DR

The initial objective in this work was to show how simple large-scale imitation learning of high-quality heuristic search methods can lead to state-of-the-art ML MAPF performance, but it is found that, at least with the model architecture, simple large-scale imitation learning does not produce impressive results.

Abstract

Multi-Agent Path Finding (MAPF) is the problem of effectively finding efficient collision-free paths for a group of agents in a shared workspace. The MAPF community has largely focused on developing high-performance heuristic search methods. Recently, several works have applied various machine learning (ML) techniques to solve MAPF, usually involving sophisticated architectures, reinforcement learning techniques, and set-ups, but none using large amounts of high-quality supervised data. Our initial objective in this work was to show how simple large scale imitation learning of high-quality heuristic search methods can lead to state-of-the-art ML MAPF performance. However, we find that, at least with our model architecture, simple large scale (700k examples with hundreds of agents per example) imitation learning does \textit{not} produce impressive results. Instead, we find that by using prior work that post-processes MAPF model predictions to resolve 1-step collisions (CS-PIBT), we can train a simple ML MAPF model in minutes that dramatically outperforms existing ML MAPF policies. This has serious implications for all future ML MAPF policies (with local communication) which currently struggle to scale. In particular, this finding implies that future learnt policies should (1) always use smart 1-step collision shields (e.g. CS-PIBT), (2) always include the collision shield with greedy actions as a baseline (e.g. PIBT) and (3) motivates future models to focus on longer horizon / more complex planning as 1-step collisions can be efficiently resolved.

Figures (7)

• Figure 1: Left: From a diverse sets of map, we can use existing strong (centralized) heuristic search solvers to solve instances with hundreds of agents. This can easily lead to thousands of MAPF solutions. Each solution contains a sequence of timesteps, where each timestep can be viewed as a "graph" of agents where each agent has a label. Middle: We can then train a local decentralized policy that takes as input a local field of view (FoV) and tries to predict the action label. Additionally, the agent can communicate to agents nearby within the FoV. Right: With naive collision shielding, our performance is not impressive. With CS-PIBT, we find that we do not need large data and a small scale imitation learning is able to get good performance.
• Figure 2: We plot the success rate, per-agent solution cost, and runtime for each method. The first row maps are seen during training, but evaluated with new start-goal positions, while the bottom maps are unseen maps. We see how our SSIL with CS-PIBT significantly outperforms EPH. It is also on-par with PIBT and EECBS with performance dependent on the map.
• Figure 3: We see how SSIL produces solutions with better solution quality when trained on data collected from EECBS run with lower suboptimality.
• Figure A1: Success, solution cost, and runtime for city and game maps ($^*$ denotes test maps).
• Figure A2: Success, solution cost, and runtime for warehouse and random maps ($^*$ denotes test maps).