Task and Motion Planning for Execution in the Real

TL;DR

Task and Motion Planning under incomplete world knowledge is addressed by introducing TAMPER, a framework that interleaves planning and execution to handle partially grounded symbolic actions. The planner yields a partially grounded task and motion plan $\overline{\mathbf{T}}$ with gaps $\pi_i=\emptyset$ that are then resolved online by executing closed-loop behaviors drawn from a supplied set $\mathcal{B}$ and by applying a repair strategy to rejoin to subsequent trajectories. Execution feedback updates symbolic constraints and may trigger replanning, enabling robust real-world task completion. The approach is validated on 40 real-world trials and a grocery demo, showing faster execution, fewer actions, and higher success in partially grounded scenarios compared to a strong replanning baseline, and a dataset is shared for reproducibility. The work broadens the applicability of TAMP to extensively grounded problems by leveraging execution-time behaviors while preserving planning-level reasoning.

Abstract

Task and motion planning represents a powerful set of hybrid planning methods that combine reasoning over discrete task domains and continuous motion generation. Traditional reasoning necessitates task domain models and enough information to ground actions to motion planning queries. Gaps in this knowledge often arise from sources like occlusion or imprecise modeling. This work generates task and motion plans that include actions cannot be fully grounded at planning time. During execution, such an action is handled by a provided human-designed or learned closed-loop behavior. Execution combines offline planned motions and online behaviors till reaching the task goal. Failures of behaviors are fed back as constraints to find new plans. Forty real-robot trials and motivating demonstrations are performed to evaluate the proposed framework and compare against state-of-the-art. Results show faster execution time, less number of actions, and more success in problems where diverse gaps arise. The experiment data is shared for researchers to simulate these settings. The work shows promise in expanding the applicable class of realistic partially grounded problems that robots can address.

Figures (14)

• Figure 1: The task is to move three objects to the goal tray. The full states of the dotted objects are not known precisely. A task and motion plan will have gaps which can only be filled in during execution, e.g., the blue object's pose can only be known after opening the drawer. The problem imposes constraints visualized in a block-world-like diagram at the lower part of the figure.
• Figure 2: The source of gaps can be diverse, including (from left to right) occlusions, imprecise simulations, and object modeling gaps (two paper bags look the same but contain different groceries).
• Figure 3: This work introduces behaviors to bridge gaps in the task and motion plan during execution time, and recovers constraints from the task, motion, and execution domains.
• Figure 4: The figure is a visual representation of grounding task and motion plans during planning and execution. The top row shows the task and motion plans. The middle visualizes the corresponding trajectories $\pi_i$ in each configuration space. The bottom shows the corresponding trajectories in the real-world during execution. The colored spaces (yellow and green) represent individual symbolic states. a) In the fully grounded case, the real world closely matches the planning model, which is known. The is typical to classical TAMP methods. b) In the partially grounded case, the light gray regions indicate that we do not have good enough knowledge to ground action $a_0$. With a typical TAMP formulation, we cannot even plan the motion for $a_0$ due to the gap. With our work, we propose to leave a gap in the task and motion plan ($\langle a_0, \emptyset \rangle$, which is filled by a behavior $\mathbf{b}$ during execution (represented by the orange circle and dotted line).
• Figure 5: The TAMPER framework.