Implicit Physics-aware Policy for Dynamic Manipulation of Rigid Objects via Soft Body Tools

TL;DR

This paper introduces the Implicit Physics-aware (IPA) policy for one-shot dynamic manipulation of rigid objects using soft-bodied tools, addressing unobservable physics and heterogeneous system dynamics. IPA operates in two stages: a SysID phase that uses a brief high-acceleration action to implicitly encode environmental physics into a trajectory, and a subsequent action-prediction phase that yields a one-shot velocity-based action $\hat{a}$. Trained in a self-supervised, simulation-based framework across diverse physics settings, IPA generalizes to real-world scenarios and outperforms baselines in simulated and physical experiments, with sim-to-real results indicating robust but imperfect transfer due to ground-plane variability. The work highlights fast adaptation to unknown physics, reduced labeling needs, and practical potential for agile, rope-based manipulation in uncertain environments.

Abstract

Recent advancements in robot tool use have unlocked their usage for novel tasks, yet the predominant focus is on rigid-body tools, while the investigation of soft-body tools and their dynamic interaction with rigid bodies remains unexplored. This paper takes a pioneering step towards dynamic one-shot soft tool use for manipulating rigid objects, a challenging problem posed by complex interactions and unobservable physical properties. To address these problems, we propose the Implicit Physics-aware (IPA) policy, designed to facilitate effective soft tool use across various environmental configurations. The IPA policy conducts system identification to implicitly identify physics information and predict goal-conditioned, one-shot actions accordingly. We validate our approach through a challenging task, i.e., transporting rigid objects using soft tools such as ropes to distant target positions in a single attempt under unknown environment physics parameters. Our experimental results indicate the effectiveness of our method in efficiently identifying physical properties, accurately predicting actions, and smoothly generalizing to real-world environments. The related video is available at: https://youtu.be/4hPrUDTc4Rg?si=WUZrT2vjLMt8qRWA

Figures (3)

• Figure 1: The workflow of the IPA policy for object transport task. IPA starts with implicitly identifying the related physical properties by performing a predefined action and recording the moving trajectory of the object. Next, the framework observes the environment to obtain the depth map and segmentation map encoding the environment and task configurations. Then, the IPA policy takes as input the aforementioned data to output a suitable action.
• Figure 2: Evaluation of SysID impact across different friction coefficients. The left plot compares the Vel-Diff metric of all methods across different friction coefficients. IPA class methods maintain a consistently low Vel-Diff, while other methods without SysID only perform well under certain friction coefficient values. The right figure presents scenarios with high and low friction coefficients. In both cases, IPA successfully transported the object to the target position. In contrast, IPA w/o SysID fails by overshooting and undershooting in low and high friction settings.
• Figure 3: An example of the real-world experiment. In this scenario, we vary the friction by putting the box into a baking tray. According to the first and third rows, our IPA policy can capture the change and adjust the casting action accordingly. In contrast, the IPA w/o SysID baseline is unable to predict an appropriate action to finish the task.