Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Factored Task and Motion Planning with Combined Optimization, Sampling and Learning

Joaquim Ortiz-Haro

TL;DR

This thesis tackles Task and Motion Planning (TAMP) by introducing a refined factored representation (Factored-NLP) that jointly handles discrete task decisions and continuous trajectory constraints. It presents two bidirectional solvers: Diverse Planning for Logic Geometric Programming (LGP) and the Factored-NLP Planner, which extract infeasible constraint subsets and feed this information back into discrete planning to prune branches. A second strand develops meta-solvers that adaptively combine sampling and optimization, using a TAMP computation tree to reason about compute operations and problem decomposition, with empirical demonstrations on Blocks, Hanoi, and Push-like domains. A third strand integrates learning to accelerate computations, including deep generative models for constraint sampling and graph neural networks to predict infeasibility, enabling faster warm-starts and feasibility checks. Together, these contributions form a cohesive framework for scalable, learning-augmented TAMP that can generalize across multi-robot, multi-object manipulation tasks and pave the way for real-time, perception-enabled autonomous planning in complex environments.

Abstract

In this thesis, we aim to improve the performance of TAMP algorithms from three complementary perspectives. First, we investigate the integration of discrete task planning with continuous trajectory optimization. Our main contribution is a conflict-based solver that automatically discovers why a task plan might fail when considering the constraints of the physical world. This information is then fed back into the task planner, resulting in an efficient, bidirectional, and intuitive interface between task and motion, capable of solving TAMP problems with multiple objects, robots, and tight physical constraints. In the second part, we first illustrate that, given the wide range of tasks and environments within TAMP, neither sampling nor optimization is superior in all settings. To combine the strengths of both approaches, we have designed meta-solvers for TAMP, adaptive solvers that automatically select which algorithms and computations to use and how to best decompose each problem to find a solution faster. In the third part, we combine deep learning architectures with model-based reasoning to accelerate computations within our TAMP solver. Specifically, we target infeasibility detection and nonlinear optimization, focusing on generalization, accuracy, compute time, and data efficiency. At the core of our contributions is a refined, factored representation of the trajectory optimization problems inside TAMP. This structure not only facilitates more efficient planning, encoding of geometric infeasibility, and meta-reasoning but also provides better generalization in neural architectures.

Factored Task and Motion Planning with Combined Optimization, Sampling and Learning

TL;DR

This thesis tackles Task and Motion Planning (TAMP) by introducing a refined factored representation (Factored-NLP) that jointly handles discrete task decisions and continuous trajectory constraints. It presents two bidirectional solvers: Diverse Planning for Logic Geometric Programming (LGP) and the Factored-NLP Planner, which extract infeasible constraint subsets and feed this information back into discrete planning to prune branches. A second strand develops meta-solvers that adaptively combine sampling and optimization, using a TAMP computation tree to reason about compute operations and problem decomposition, with empirical demonstrations on Blocks, Hanoi, and Push-like domains. A third strand integrates learning to accelerate computations, including deep generative models for constraint sampling and graph neural networks to predict infeasibility, enabling faster warm-starts and feasibility checks. Together, these contributions form a cohesive framework for scalable, learning-augmented TAMP that can generalize across multi-robot, multi-object manipulation tasks and pave the way for real-time, perception-enabled autonomous planning in complex environments.

Abstract

In this thesis, we aim to improve the performance of TAMP algorithms from three complementary perspectives. First, we investigate the integration of discrete task planning with continuous trajectory optimization. Our main contribution is a conflict-based solver that automatically discovers why a task plan might fail when considering the constraints of the physical world. This information is then fed back into the task planner, resulting in an efficient, bidirectional, and intuitive interface between task and motion, capable of solving TAMP problems with multiple objects, robots, and tight physical constraints. In the second part, we first illustrate that, given the wide range of tasks and environments within TAMP, neither sampling nor optimization is superior in all settings. To combine the strengths of both approaches, we have designed meta-solvers for TAMP, adaptive solvers that automatically select which algorithms and computations to use and how to best decompose each problem to find a solution faster. In the third part, we combine deep learning architectures with model-based reasoning to accelerate computations within our TAMP solver. Specifically, we target infeasibility detection and nonlinear optimization, focusing on generalization, accuracy, compute time, and data efficiency. At the core of our contributions is a refined, factored representation of the trajectory optimization problems inside TAMP. This structure not only facilitates more efficient planning, encoding of geometric infeasibility, and meta-reasoning but also provides better generalization in neural architectures.
Paper Structure (193 sections, 5 theorems, 48 equations, 72 figures, 7 tables, 7 algorithms)

This paper contains 193 sections, 5 theorems, 48 equations, 72 figures, 7 tables, 7 algorithms.

Table of Contents

  1. Acknowledgements
  2. Abstract
  3. Zusammenfassung
  4. Resumen
  5. Resum
  6. Introduction
  7. Autonomy of robotic systems
  8. Task and Motion Planning
  9. Sampling and Optimization Methods for Task and Motion Planning
  10. Meta-solvers: An adaptive combination of sampling and optimization
  11. Accelerating Model-Based Solvers with Deep Learning
  12. The Factored Structure of Task and Motion Planning
  13. Reading Guide and Statement of Contributions
  14. Background
  15. Nonlinear Programs in Robotics
  16. ...and 178 more sections

Key Result

Theorem 9.1

Let $\pi = \langle a_1, \ldots, a_K \rangle$ be a sequence of discrete actions, such that $\pi$ is not geometrically feasible from the initial state. Then any sequence of actions $\pi'$ which contains $\pi$ as a prefix is not geometrically feasible from the initial state.

Figures (72)

  • Figure 1: Task and Motion Planning -- Example 1.0.
  • Figure 2: A solution to the Task and Motion Planning problem of \ref{['fig:intro:example1']}.
  • Figure 3: Task and Motion Planning -- Example 2.0.
  • Figure 4: A solution to the Task and Motion Planning problem in \ref{['fig:intro:example2']}.
  • Figure 5: Reading Guide -- A graphical overview.
  • ...and 67 more figures

Theorems & Definitions (10)

  • Definition 9.1
  • Theorem 9.1
  • proof
  • Definition 9.2
  • Theorem 9.2
  • proof
  • Definition 10.1
  • Proposition 10.1
  • Theorem 10.1
  • Theorem 12.1: Existence of a Search Algorithm in the TAMP Computation Tree