Multi-Agent Monte Carlo Tree Search for Makespan-Efficient Object Rearrangement in Cluttered Spaces

TL;DR

This work tackles makespan-efficient object rearrangement in cluttered spaces with both monotone and non-monotone tasks. It introduces CAM-MCTS, a centralized, asynchronous Monte Carlo Tree Search framework that couples centralized task assignment with an asynchronous execution strategy to avoid idle time and synchronization delays. Key contributions include a centralized task assignment module, an asynchronous execution mechanism with one-step look-ahead, and a parallelizable MCTS-based planner that yields superior makespan, success rate, and scalability in both simulation and real-robot experiments. The results demonstrate practical impact for multi-robot logistics, warehouse automation, and search-and-rescue tasks by enabling efficient, scalable planning under complex environmental constraints.

Abstract

Object rearrangement planning in complex, cluttered environments is a common challenge in warehouses, households, and rescue sites. Prior studies largely address monotone instances, whereas real-world tasks are often non-monotone-objects block one another and must be temporarily relocated to intermediate positions before reaching their final goals. In such settings, effective multi-agent collaboration can substantially reduce the time required to complete tasks. This paper introduces Centralized, Asynchronous, Multi-agent Monte Carlo Tree Search (CAM-MCTS), a novel framework for general-purpose makespan-efficient object rearrangement planning in challenging environments. CAM-MCTS combines centralized task assignment-where agents remain aware of each other's intended actions to facilitate globally optimized planning-with an asynchronous task execution strategy that enables agents to take on new tasks at appropriate time steps, rather than waiting for others, guided by a one-step look-ahead cost estimate. This design minimizes idle time, prevents unnecessary synchronization delays, and enhances overall system efficiency. We evaluate CAM-MCTS across a diverse set of monotone and non-monotone tasks in cluttered environments, demonstrating consistent reductions in makespan compared to strong baselines. Finally, we validate our approach on a real-world multi-agent system under different configurations, further confirming its effectiveness and robustness.

Figures (5)

• Figure 1: The figure illustrates a non-monotone four-object relocation task. The left image shows the objects' start and goal states, while the others depict key pick-and-place robot actions, leading to the desired object rearrangement configuration.
• Figure 2: The figure illustrates the asynchronous task execution design for agent A1 to A4. Starting from the synchronous task execution (a), our method leverges the cost estimation with one-step look-ahead (b) to find the most appropriate time step $t'_a$ (c) to terminate the current iteration. In this way, agent A1 and A2 can start performing new tasks without waiting for A3 and A4.
• Figure 3: This figure depicts the narrow passage, the warehouse, the random obstacles and the maze environments. The colored cubes are the objects while the block ones are the robots.
• Figure 4: These results compare CAM-MCTS, CAM-UCS, and RSP only in scenarios where all methods succeed, and should be interpreted in conjunction with Table 1. As shown, CAM-UCS achieves slightly better makespan in tasks with fewer objects, but only by a narrow margin over CAM-MCTS—and at the cost of significantly higher planning times. RSP, on the other hand, achieves low planning times but suffers from the highest makespan. Taken together, these results highlight that CAM-MCTS provides the best overall trade-off between planning time, makespan, and scalability to more challenging scenarios.
• Figure 5: The figure shows a non-monotone four objects counterclockwise shuffling case. The start and goal configurations are shown on the left, with the red and green arrows indicating the goal location for each object. The rest of the images denote the keyframes involving object pick and place. Our CAM-MCTS finds the asynchronous solutio in 0.83 seconds, while the actual execution lasts 394 seconds.