ScheduleStream: Temporal Planning with Samplers for GPU-Accelerated Multi-Arm Task and Motion Planning & Scheduling

TL;DR

ScheduleStream addresses the challenge of enabling parallel arm motion in Task and Motion Planning by introducing a domain-independent, Python-based language that combines instantaneous and durative actions with samplers (streams). It presents eager, lazy, and related scheduling algorithms that alternate between scheduling and sampling, and extends the framework with GPU-accelerated streams to batch collision checks, IK, and motion planning. Through simulated experiments and real-world demonstrations on multi-arm platforms, ScheduleStream yields higher success rates and shorter makespans than sequential baselines, and scales better with parallelism thanks to GPU acceleration. The work contributes the ScheduleStream language, lazy scheduling strategy, and GPU-accelerated sampling for TAMPAS, offering a practical path to efficient, temporally coordinated multi-arm manipulation.

Abstract

Bimanual and humanoid robots are appealing because of their human-like ability to leverage multiple arms to efficiently complete tasks. However, controlling multiple arms at once is computationally challenging due to the growth in the hybrid discrete-continuous action space. Task and Motion Planning (TAMP) algorithms can efficiently plan in hybrid spaces but generally produce plans, where only one arm is moving at a time, rather than schedules that allow for parallel arm motion. In order to extend TAMP to produce schedules, we present ScheduleStream, the first general-purpose framework for planning & scheduling with sampling operations. ScheduleStream models temporal dynamics using hybrid durative actions, which can be started asynchronously and persist for a duration that's a function of their parameters. We propose domain-independent algorithms that solve ScheduleStream problems without any application-specific mechanisms. We apply ScheduleStream to Task and Motion Planning & Scheduling (TAMPAS), where we use GPU acceleration within samplers to expedite planning. We compare ScheduleStream algorithms to several ablations in simulation and find that they produce more efficient solutions. We demonstrate ScheduleStream on several real-world bimanual robot tasks at https://schedulestream.github.io.

Figures (6)

• Figure 1: Real-World Demonstration. Using ScheduleStream, a bimanual robot solves for a schedule to sort the apple into the red bin and the lime in the green bin. ScheduleStream algorithms automatically select which arm to use for which object based on kinematics and execute the actions asynchronously in parallel.
• Figure 2: Illustrative Examples. Three example bimanual TAMPAS problems where the goal is for arm pythonarm1 to hold object pythonobj1 and arm pythonarm2 to hold object pythonobj2. Problem 1 ( left): the arms can pick their objects in parallel. Problem 2 ( center): the arms can only pick their objects sequentially. Problem 3 ( right): one arm must retreat after picking its object in order to make space for the other.
• Figure 3: Example Schedules.Left: a solution schedule for Example 1 in Figure \ref{['fig:bimanual']}. Each arm is able to execute a move action in parallel because the trajectories python"@t1", python"@t2" do not collide. Right: a solution schedule for Example 2. Although trajectories python"@t1", python"@t2" collide, the trajectory-configuration pairs python"@t1", pythonq2 and python"@t2", python"@q1" do not, admitting a serial solution that has a larger makespan.
• Figure 4: Simulated Experiments. The "Franka Hold Any 4", "SO100 Hold Any 4", "Franka Pack 4", and "Franka Stack 4" tasks.
• Figure I: Success Rate and First Solution Time. Hierarchical fails to solve several tasks because it does not backtrack, Ours has a lower success rate and higher runtime than its GPU-accelerated counterpart Ours + GPU.

Theorems & Definitions (2)

• Definition 1
• Definition 2