Object-Centric Kinodynamic Planning for Nonprehensile Robot Rearrangement Manipulation

TL;DR

This work tackles large-scale, nonprehensile rearrangement by introducing an object-centric planning framework that decouples object motion from robot actions. It plans desired object trajectories using an object-centric, sampling-based planner with two complementary exploration modes, then realizes those trajectories online via a closed-loop pushing strategy (UNO Push) on a bounded workspace. Interleaved planning and execution mitigate real-world uncertainties and modeling inaccuracies, enabling robust performance in simulation and on a real 7-DoF robot, and it provides a standardized benchmark protocol for future research. The approach improves planning efficiency and task effectiveness across diverse tasks, including those without explicit goals, and demonstrates generalization to non-convex objects and varied object classes.

Abstract

Nonprehensile actions such as pushing are crucial for addressing multi-object rearrangement problems. Many traditional methods generate robot-centric actions, which differ from intuitive human strategies and are typically inefficient. To this end, we adopt an object-centric planning paradigm and propose a unified framework for addressing a range of large-scale, physics-intensive nonprehensile rearrangement problems challenged by modeling inaccuracies and real-world uncertainties. By assuming each object can actively move without being driven by robot interactions, our planner first computes desired object motions, which are then realized through robot actions generated online via a closed-loop pushing strategy. Through extensive experiments and in comparison with state-of-the-art baselines in both simulation and on a physical robot, we show that our object-centric planning framework can generate more intuitive and task-effective robot actions with significantly improved efficiency. In addition, we propose a benchmarking protocol to standardize and facilitate future research in nonprehensile rearrangement.

Figures (21)

• Figure 1: Through object-centric planning, our framework is able to efficiently rearrange multiple movable objects of different shapes to accomplish various tasks. In the scene, "T", "R", and "O" letter-shaped objects are rearranged to form the abbreviation "TRO".
• Figure 2: Qualitative comparison of the state-of-the-art rearrangement solutions. Each column represents one characteristic of the solution, from left to right: 1) Object-Centric: the proposed method incorporates object-centric components (e.g., sampling strategy, action primitives, planning paradigm, etc), otherwise, robot-centric; 2) Nonprehensile: nonprehensile actions are incorporated; 3) Data-Driven: the method is data-driven and requires extra time for training; 4) Closed-Loop: the method generates closed-loop motion plans that can handle real-world uncertainties; 5) Versatile: the method has shown the capability to transfer across different rearrangement tasks (e.g., relocating, separating, sorting, etc); 6) Explicit-Goal: the method requires an explicit goal pose or location for each object. 7) Max # Objects: the maximum number of movable objects the method can deal with, as shown by the corresponding (simulation and real-world) experiments.
• Figure 3: Left: A robot pushing action is represented by $\bm{u}_t = \left(\alpha_t, \beta_t\right)$, where $\alpha_t$ and $\beta_t$ are two angles specified in the object's body frame. To perform $\bm{u}_t$, the robot needs to first place its pusher at the position $P_t$ (determined by $\alpha_t$) and then translate in the direction of $\beta_t$ by a distance $d_{push}$ to interact with the object (blue cube) through the push (orange arrow); Right: By consecutively generating and executing the pushing actions $\bm{u}_0$ through $\bm{u}_3$, the robot can push the object to follow a desired reference trajectory (yellow dashed lines), as a sequence of desired poses or positions of the object. With a closed-loop pushing strategy, the object's actual trajectory (green dashed lines) due to execution will not deviate much from the reference.
• Figure 4: A schematic plot of OCP. In the left figure, a motion tree is progressively grown from the start arrangement (lower left) towards the goal arrangement (upper right). Each edge of the tree is an explored object motion (i.e., trajectory) generated by one of two exploration modes, Mode I (blue line) and Mode II (red line). Each edge leads to an outcome arrangement represented by a tree node. Replanning (green dots) by sensing real-world arrangement is needed to eliminate the errors between the planned rearrangement solution (solid blue and red lines) and real execution (green lines) due to real-world uncertainties. The right figure shows the planned rearrangement since the last replanning, which consists of two consecutive object motions. The first (solid red line) is an Mode II motion that moves the red cube through a curvy Trajectory I (orange), resulting in an arrangement shown by Arrangement II; the second motion (solid blue line) is under Mode I, and leads to the arrangement shown in Arrangement III by moving the blue cube through a straight-line Trajectory II (orange).
• Figure 5: Left: A scenario of the object sorting task. Two classes of cubes (red and blue) need to be relocated inside their corresponding goal regions (circles in the same color as the cubes). At the current state, all cubes except for Cube $\# 2$ are already sorted. Middle: The value of $f\left(\lvert\nabla_{\bm{s}^j} h(\bm{s}) \rvert \right)$ for each object, if directly used as the sampling probability, will cause the algorithm trapped by keeping sampling Cube $\# 2$ to activate. Since there is no free space around the unsorted Cube $\# 2$ for the robot to approach it, the algorithm will be likely stuck at this point. Right: Enabled by the weighted mixture of Gaussians, the sampling probabilities of the red cubes ($\# 0$, $\# 1$, $\# 3$, and $\# 4$) surrounding Cube $\# 2$ are increased so that they can be moved to create some free space for Cube $\# 2$, to facilitate the relocation of Cube $\# 2$ to its goal region by the subsequent actions.