Beyond Task and Motion Planning: Hierarchical Robot Planning with General-Purpose Skills

TL;DR

The paper addresses long-horizon robot planning when high-level actions involve non-kinematic, contact-rich skills. It introduces Task and Skill Planning (TASP), a hierarchical framework that wraps general-purpose object-centric skills with free-space motion via Composable Interaction Primitives (CIPs) to enable composition and planning. By integrating CIP-based skills into an ATAM-based refinement, the method achieves efficient long-horizon planning validated on a bimanual manipulator and a mobile manipulator performing tasks such as erasing, spreading, opening doors, and navigating environments. The approach broadens TAMP applicability to real-world manipulation with force feedback and sustained contacts, with potential impact on robust, scalable robot operation in unstructured settings.

Abstract

Task and motion planning is a well-established approach for solving long-horizon robot planning problems. However, traditional methods assume that each task-level robot action, or skill, can be reduced to kinematic motion planning. We address the challenge of combining motion planning with closed-loop motor controllers that go beyond mere kinematic considerations. We propose a novel framework that integrates these policies into motion planning using Composable Interaction Primitives (CIPs), enabling the use of diverse, non-composable pre-learned skills in hierarchical robot planning. We validate our Task and Skill Planning (TASP) approach through real-world experiments on a bimanual manipulator and a mobile manipulator, demonstrating that CIPs allow diverse robots to combine motion planning with general-purpose skills to solve complex, long-horizon tasks.

Figures (4)

• Figure 1: A Boston Dynamics Spot and a bimanual manipulator use task and skill planning (TASP) to solve long-horizon hybrid robot planning problems. On the left, the robots execute motion-planning-based skills to pick up an eraser (Fig. \ref{['fig:1a-pick']}) and a jar of peanut butter (Fig. \ref{['fig:1c']}). On the right, the robots use general-purpose skills to maintain force-controlled contact while erasing a whiteboard (Fig. \ref{['fig:1b']}) and spreading peanut butter (Fig. \ref{['fig:1d']}).
• Figure 2: Example keyframes showing the environment state before, during, and after the Erase(?board) force-controlled skill. In Fig. \ref{['fig:erase-1']}, the Spot robot can reach the whiteboard, but its gripper is stowed. Therefore, the robot is not in a configuration that fulfills the initiation condition for the Erase(?board) skill. Fig. \ref{['fig:erase-2']} shows the state after Spot has executed a head motion plan to reach a configuration that fulfills the initiation condition for the Erase(?board) skill. Fig. \ref{['fig:erase-3']} depicts a state during the execution of the Erase(?board) skill. Finally, Fig. \ref{['fig:erase-4']} shows the state after Spot has executed a tail motion plan to stow its arm, resulting in a configuration that is no longer in the termination set of the Erase(?board) skill.
• Figure 3: We conduct real-world experiments on two robot platforms to demonstrate the versatility of our task and skill planning approach. In our bimanual manipulation setting, the robot can pick up a jar and a kitchen knife, as shown in Fig. \ref{['fig:dorfl-grasp-pb']} and Fig. \ref{['fig:dorfl-grasp-knife']}, using a Grasp(?object) skill implemented using motion planning. The bimanual manipulator has three additional skills that combine trajectory playback and impedance control: in Fig. \ref{['fig:dorfl-open-pb']}, the robot uses Open(?jar) to open a jar of peanut butter using both grippers; Fig. \ref{['fig:dorfl-scoop']} shows the robot scooping peanut butter onto a knife using its Scoop(?knife,?jar) skill; and Fig. \ref{['fig:dorfl-spread']} depicts the robot executing a Spread(?knife,?bread) skill to spread peanut butter.
• Figure 4: We also evaluate our method using a multi-room mobile manipulation experiment that requires long-horizon hybrid robot planning. Fig. \ref{['fig:results-open-door']} shows the Spot opening a door using OpenDoor(?door), an off-the-shelf skill built into the robot. Fig. \ref{['fig:results-open-drawer']} and Fig. \ref{['fig:results-close-door']} depict Spot using trajectory playback skills, OpenDrawer(?drawer) and CloseDoor(?door), to open a drawer and close a door, respectively. Fig. \ref{['fig:results-go-to']} and Fig. \ref{['fig:results-place']} show Spot using motion-planning-based skills to navigate to an adjacent room using a GoTo(?location) skill and place an eraser on top of a filing cabinet using Place(?object,?surface). Finally, Fig. \ref{['fig:results-erase']} shows Spot erasing a whiteboard using a force-controlled skillErase(?board).

Theorems & Definitions (4)

• Definition 1
• Definition 2
• Definition 3
• Definition 4