Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Stage Monte Carlo Tree Search for Non-Monotone Object Rearrangement Planning in Narrow Confined Environments

Hanwen Ren, Ahmed H. Qureshi

TL;DR

The paper tackles non-monotone object rearrangement in narrow confined environments, a problem that is NP-hard due to joint object ordering and relocation constraints. It introduces a Multi-Stage Monte Carlo Tree Search (MS-MCTS) framework that decouples the problem into ordered subproblems via an object stage topology and a subgoal-focused expansion strategy, augmented by a linear motion planner to minimize swept volume. Key contributions include the object stage topology generation, a subgoal-focused SS-MCTS with blocking-object handling, and post-optimization that merges actions for efficiency, all validated by sim-to-real experiments on a UR5e. The results demonstrate superior planning time and plan quality compared to baselines, showing practical impact for efficient, high-quality object rearrangement in tight spaces.

Abstract

Non-monotone object rearrangement planning in confined spaces such as cabinets and shelves is a widely occurring but challenging problem in robotics. Both the robot motion and the available regions for object relocation are highly constrained because of the limited space. This work proposes a Multi-Stage Monte Carlo Tree Search (MS-MCTS) method to solve non-monotone object rearrangement planning problems in confined spaces. Our approach decouples the complex problem into simpler subproblems using an object stage topology. A subgoal-focused tree expansion algorithm that jointly considers the high-level planning and the low-level robot motion is designed to reduce the search space and better guide the search process. By fitting the task into the MCTS paradigm, our method produces optimistic solutions by balancing exploration and exploitation. The experiments demonstrate that our method outperforms the existing methods in terms of the planning time, the number of steps, and the total move distance. Moreover, we deploy our MS-MCTS to a real-world robot system and verify its performance in different scenarios.

Multi-Stage Monte Carlo Tree Search for Non-Monotone Object Rearrangement Planning in Narrow Confined Environments

TL;DR

The paper tackles non-monotone object rearrangement in narrow confined environments, a problem that is NP-hard due to joint object ordering and relocation constraints. It introduces a Multi-Stage Monte Carlo Tree Search (MS-MCTS) framework that decouples the problem into ordered subproblems via an object stage topology and a subgoal-focused expansion strategy, augmented by a linear motion planner to minimize swept volume. Key contributions include the object stage topology generation, a subgoal-focused SS-MCTS with blocking-object handling, and post-optimization that merges actions for efficiency, all validated by sim-to-real experiments on a UR5e. The results demonstrate superior planning time and plan quality compared to baselines, showing practical impact for efficient, high-quality object rearrangement in tight spaces.

Abstract

Non-monotone object rearrangement planning in confined spaces such as cabinets and shelves is a widely occurring but challenging problem in robotics. Both the robot motion and the available regions for object relocation are highly constrained because of the limited space. This work proposes a Multi-Stage Monte Carlo Tree Search (MS-MCTS) method to solve non-monotone object rearrangement planning problems in confined spaces. Our approach decouples the complex problem into simpler subproblems using an object stage topology. A subgoal-focused tree expansion algorithm that jointly considers the high-level planning and the low-level robot motion is designed to reduce the search space and better guide the search process. By fitting the task into the MCTS paradigm, our method produces optimistic solutions by balancing exploration and exploitation. The experiments demonstrate that our method outperforms the existing methods in terms of the planning time, the number of steps, and the total move distance. Moreover, we deploy our MS-MCTS to a real-world robot system and verify its performance in different scenarios.
Paper Structure (17 sections, 3 figures, 2 tables, 2 algorithms)

This paper contains 17 sections, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Methodology
  5. Linear Motion Planner
  6. Object Stage Topology Generation
  7. Single-Stage MCTS planner
  8. Multi-Stage MCTS planner $\&$ Post Optimization
  9. Experiments
  10. Simulation Setup
  11. Simulation Results
  12. Ablation Studies
  13. Continuous buffer region proposal
  14. Object stage topology order
  15. Post optimization
  16. ...and 2 more sections

Figures (3)

  • Figure 1: The figure shows a four objects flipping case with complex object dependency relations. Our method solves it in 8 steps without redundant robot actions.
  • Figure 2: (a) The gripper at $p_m$ uses the collision-free picking path (blue) and placing path (red) generated from our linear motion planner to relocate the object at $p^t_i$. Objects with a black contour are already placed at their goal regions. (b) The left sub-figure shows the order from the longitude heuristic while the right one illustrates the generation process of the dependency graph. For example, the goal region of the grey object blocks the turquoise, blue, orange, and pink objects, which creates four edges toward the grey object in the dependency graph. The final object stage topology is created by jointly considering the longitude heuristic and the dependency graph.
  • Figure 3: Our MS-MCTS method has higher success rates and fewer steps than the baselines across all difficulty levels.