Adaptive Anytime Multi-Agent Path Finding Using Bandit-Based Large Neighborhood Search

TL;DR

BALANCE addresses the scalability bottleneck in Anytime MAPF by introducing a bi-level bandit framework that adaptively selects destroy heuristics and neighborhood sizes during Large Neighborhood Search. The top-level bandit chooses a destroy heuristic $H\in\mathcal{H}$, while a bottom-level bandit per $H$ selects an exponent $e$ to determine $N=2^{e}$, with rewards defined as the cost improvement $x_k=\max\{0,c(P)-c(P^{+})\}$. Instantiations using Roulette Wheel, UCB1, and Thompson Sampling demonstrate that online, reward-driven adaptation significantly improves performance across benchmark maps, with BALANCE-TS often yielding the best results and reducing dependence on prior tuning. Across large-scale MAPF scenarios, BALANCE achieves cost improvements of at least 50% over state-of-the-art baselines, illustrating strong practical impact and the viability of online bandit-based adaptation for complex multi-agent planning.

Abstract

Anytime multi-agent path finding (MAPF) is a promising approach to scalable path optimization in large-scale multi-agent systems. State-of-the-art anytime MAPF is based on Large Neighborhood Search (LNS), where a fast initial solution is iteratively optimized by destroying and repairing a fixed number of parts, i.e., the neighborhood, of the solution, using randomized destroy heuristics and prioritized planning. Despite their recent success in various MAPF instances, current LNS-based approaches lack exploration and flexibility due to greedy optimization with a fixed neighborhood size which can lead to low quality solutions in general. So far, these limitations have been addressed with extensive prior effort in tuning or offline machine learning beyond actual planning. In this paper, we focus on online learning in LNS and propose Bandit-based Adaptive LArge Neighborhood search Combined with Exploration (BALANCE). BALANCE uses a bi-level multi-armed bandit scheme to adapt the selection of destroy heuristics and neighborhood sizes on the fly during search. We evaluate BALANCE on multiple maps from the MAPF benchmark set and empirically demonstrate cost improvements of at least 50% compared to state-of-the-art anytime MAPF in large-scale scenarios. We find that Thompson Sampling performs particularly well compared to alternative multi-armed bandit algorithms.

Figures (6)

• Figure 1: Bi-level multi-armed bandit scheme of BALANCE. The top-level $\mathcal{H}$-Bandit selects a destroy heuristic $H \in \mathcal{H}$. Each bottom-level $\mathcal{N}$-Bandit corresponds to a destroy heuristic choice and selects an exponent $e \in \mathcal{N} = \{1,...,E\}$ to determine the neighborhood size $N = 2^{e}$.
• Figure 2: Sum of delays for different BALANCE variants with different time budgets compared with the respective empirically best choices. Shaded areas show the 95% confidence interval. The legend at the top applies across all plots.
• Figure 3: Relative frequencies of selected destroy heuristic and neighborhood size combinations $\langle H, N \rangle$ per BALANCE variant after 128 seconds of planning. Magenta dashed boxes indicate the empirically best destroy heuristic and neighborhood size.
• Figure 4: MAB choices over time for ost003d.
• Figure 5: Sum of delays for different BALANCE variants with different neighborhood size options $E$ and numbers of agents $m$. The time budget is 128 seconds. Shaded areas show the 95% confidence interval. The legend at the top applies across all plots.