Incremental Few-Shot Adaptation for Non-Prehensile Object Manipulation using Parallelizable Physics Simulators

TL;DR

The paper tackles the challenge of rapid adaptation for non-prehensile object manipulation by integrating an incremental, few-shot parameter optimization loop with a parallelizable physics simulator (MuJoCo) inside model-predictive control. It jointly optimizes simulator dynamics parameters and plans smooth, low-rate keypoint trajectories via minimum snap trajectories, using cross-entropy method to update both planning and model parameters from replayed real and simulated rollouts. Demonstrations in simulation and on a real 7-DoF robot show improved parameter estimates, reduced MPC execution times, and shorter trajectories after a few episodes, validating the real2sim2real approach for object pushing. The approach enables real-time, data-efficient adaptation of dynamics for improved planning, with practical implications for open-world manipulation tasks where exact physical properties are unknown or change over time.

Abstract

Few-shot adaptation is an important capability for intelligent robots that perform tasks in open-world settings such as everyday environments or flexible production. In this paper, we propose a novel approach for non-prehensile manipulation which incrementally adapts a physics-based dynamics model for model-predictive control (MPC). The model prediction is aligned with a few examples of robot-object interactions collected with the MPC. This is achieved by using a parallelizable rigid-body physics simulation as dynamic world model and sampling-based optimization of the model parameters. In turn, the optimized dynamics model can be used for MPC using efficient sampling-based optimization. We evaluate our few-shot adaptation approach in object pushing experiments in simulation and with a real robot.

Figures (6)

• Figure 1: Our approach is twofold: Rigid-body physics simulation is utilized as a dynamics model in model predictive control (MPC) to plan target keypoints for a minimum snap Cartesian trajectory. To better match the dynamics model to the environment (e.g. real world) and thereby improve the performance of the MPC, observed robot-object interaction rollouts are consecutively stored in a replay buffer and used to optimize a subset of model parameters. Parameter optimization is iteratively executed by replaying the current buffer of trajectories and matching the resulting behavior to the ground truth. The optimized set of parameters $\boldsymbol{\theta}^*$ is then used by MPC to execute tasks. Environments: Real robot experiment setup (left) including end-effector sphere attachment and objects with motion capture markers; dynamics model/simulation setup (right) realizing robot end-effector (red sphere) and object (gray cuboid) as shape primitives.
• Figure 2: Parameter estimation errors of sliding, torsional and rolling friction for $\Delta=0.2$ (top) and $\Delta=1$ (bottom) during optimization over 10 episodes. Boxes are bounded by upper and lower quartiles. Initial error at "episode" $0$.
• Figure 3: Success rate, mean task execution time, and validation replay loss for $\Delta = 0.2$ (blue) and $\Delta = 1$ (orange).
• Figure 4: Model parameters over optimization episodes, starting from two initial sets (orange: $( \theta_{s}, \theta_{t}, \theta_m) = (1.0, 1.0, 50)$, blue: $( \theta_{s}, \theta_{t}, \theta_m) = (1.0, 0.005, 50)$).
• Figure 5: Success rate $S$, avg. execution time $\overline{t}$, avg. MPC loss over the executed trajectories $\overline{\mathcal{L}}$, avg. object trajectory length $\overline{l}_{o}$ and avg. robot end-effector trajectory length $\overline{l}_{r}$ for initial parameter sets $( \theta_{s}, \theta_{t}, \theta_m) = (1.0, 1.0, 50)$ (orange) and $( \theta_{s}, \theta_{t}, \theta_m) = (1.0, 0.005, 50)$ (blue).