Differentiable GPU-Parallelized Task and Motion Planning

TL;DR

cuTAMP introduces the first GPU-accelerated, backtracking TAMP framework that unites sampling-based initialization with differentiable, batch optimization to solve highly constrained plan-skeleton CSPs in seconds. By operating on thousands of particles in parallel, it maintains interdependencies among continuous parameters while exploring multiple solution basins and leveraging a plan-feasibility heuristic to guide skeleton selection. The method demonstrates strong performance on diverse manipulation tasks, both in simulation and on real robots (UR5 and Kinova), and shows notable improvements over serial baselines in success rate and runtime. This approach enables robust, long-horizon planning for complex, contact-rich manipulation with practical impact for real-world robotic systems.

Abstract

Planning long-horizon robot manipulation requires making discrete decisions about which objects to interact with and continuous decisions about how to interact with them. A robot planner must select grasps, placements, and motions that are feasible and safe. This class of problems falls under Task and Motion Planning (TAMP) and poses significant computational challenges in terms of algorithm runtime and solution quality, particularly when the solution space is highly constrained. To address these challenges, we propose a new bilevel TAMP algorithm that leverages GPU parallelism to efficiently explore thousands of candidate continuous solutions simultaneously. Our approach uses GPU parallelism to sample an initial batch of solution seeds for a plan skeleton and to apply differentiable optimization on this batch to satisfy plan constraints and minimize solution cost with respect to soft objectives. We demonstrate that our algorithm can effectively solve highly constrained problems with non-convex constraints in just seconds, substantially outperforming serial TAMP approaches, and validate our approach on multiple real-world robots. Project website and code: https://cutamp.github.io

Figures (25)

• Figure 1: Results on Single Object Packing. We ablate the particle batch size $N_b$, where $N_b = 1$ is representative of serial approaches garrett2018samplingtoussaint2015logic. The #Satisfying metric measures the number of satisfying particles. The best solution time for each approach is bolded, and the overall best is highlighted.
• Figure 2: Object Packing with a UR5. The objective is to place all objects onto the white region while minimizing the distance between them. The final state achieves a tight packing with successful reduction of the goal cost.
• Figure 2: Results on Bookshelf. #Opt. Plans metric measures the number of plan skeletons that were optimized or resampled before a solution was found (Stage 2 of Algorithm \ref{['alg:gpu-tamp']}).
• Figure 3: Optimizing Goal Costs. We minimize the distance between four objects and compare the best particle cost. $\lambda$ is the weight applied to the goal cost during optimization.
• Figure 4: Minimizing Distance between Objects. The state after executing the best particle. (a) cuTAMP achieves significantly lower cost compared to (b) parallelized sampling.

Theorems & Definitions (5)

• Definition 1
• Definition 2
• Definition 3
• Theorem 1
• proof