Learn2Decompose: Learning Problem Decomposition for Efficient Sequential Multi-object Manipulation Planning

TL;DR

This work tackles the exponential growth of planning time in sequential multi-object manipulation by learning problem decompositions from demonstrations. It introduces three modules—goal decomposition learning (via sequential pattern mining to extract must-pass subgoal sequences), computational distance learning (a GNN that predicts object importance to identify the closest subgoal), and object reduction (parallel replanning over reduced object sets)—all integrated with a classical TAMP solver (PDDLStream). The approach reduces planning horizon and active object counts, improving replanning speed and robustness to disturbances, and generalizes to more realistic kitchens and real-world robots. Empirical evaluations across Tower Construction, Tool Use, Cooking benchmarks, and transfers to PR2 PyBullet and real Franka experiments demonstrate significant gains over baselines and strong transferability with limited retraining.

Abstract

We present an efficient task and motion replanning approach for sequential multi-object manipulation in dynamic environments. Conventional Task And Motion Planning (TAMP) solvers experience an exponential increase in planning time as the planning horizon and number of objects grow, limiting their applicability in real-world scenarios. To address this, we propose learning problem decompositions from demonstrations to accelerate TAMP solvers. Our approach consists of three key components: goal decomposition learning, computational distance learning, and object reduction. Goal decomposition identifies the necessary sequences of states that the system must pass through before reaching the final goal, treating them as subgoal sequences. Computational distance learning predicts the computational complexity between two states, enabling the system to identify the temporally closest subgoal from a disturbed state. Object reduction minimizes the set of active objects considered during replanning, further improving efficiency. We evaluate our approach on three benchmarks, demonstrating its effectiveness in improving replanning efficiency for sequential multi-object manipulation tasks in dynamic environments.

Figures (9)

• Figure 1: The Franka Emika robot arm responds robustly to disturbances (black dashed line) in the positions of the crackers box and meat can, using our efficient task and motion replanning approach, which is enabled by integrating learned problem decomposition from demonstrations.
• Figure 2: An overview of our efficient planner, which integrates goal decomposition, computational distance, and object reduction with classical TAMP solvers to enable efficient replanning under online disturbances (orange 'x' symbol). Goal decomposition identifies the necessary subgoal sequences $(\bm{SG_{0}} \rightarrow \bm{SG_{1} \rightarrow \bm{SG_{2}}})$ offline from demonstrations. The computational distance metric is trained offline using the same set of demonstrations and is used to measure the distance (green arrows) from a disturbed state $\bm{L_{0}^{'}}$ to the subgoals (black circles), enabling the identification of the closest subgoal $\bm{SG^{*}}=\bm{SG_{0}}$. Object reduction (area with dashed line) considered grey cubes as fixed obstacles while replanning from $\bm{L_{0}^{'}}$ to $\bm{SG^{*}}$ so that only necessary objects are involved in. The whole process is repeated iteratively until the final task goal is achieved.
• Figure 3: Illustration of objects' configurations at specific time step and its graph representation.
• Figure 4: Illustration of subgoal sequences for two task goals $\bm{G}_1, \bm{G}_2$ in tower construction toy example.
• Figure 5: In (a) and (b), transparent blocks indicate a representative task goal. In the Cooking domain, the Franka manipulator is equipped with a virtual mobile base, enabling movement along the Y-axis within the boundary indicated by green lines.