PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation

TL;DR

PIN-WM addresses the challenge of learning robust non-prehensile manipulation policies with limited data by building a physics-informed 3D world model learned from RGB observations using differentiable physics and 2D Gaussian Splatting. It identifies core dynamics parameters and rendering parameters, then uses a velocity-based Linear Complementarity Problem to simulate object motion under contact and friction, enabling end-to-end differentiable learning. To bridge the Sim2Real gap, it creates physics-aware digital cousins by perturbing around the identified values, improving generalization without real-world fine-tuning. Across simulation and real-world experiments on push and flip tasks, PIN-WM with physics-aware digital cousins outperforms Real2Sim2Real baselines, demonstrating data-efficient learning and robust transfer.

Abstract

While non-prehensile manipulation (e.g., controlled pushing/poking) constitutes a foundational robotic skill, its learning remains challenging due to the high sensitivity to complex physical interactions involving friction and restitution. To achieve robust policy learning and generalization, we opt to learn a world model of the 3D rigid body dynamics involved in non-prehensile manipulations and use it for model-based reinforcement learning. We propose PIN-WM, a Physics-INformed World Model that enables efficient end-to-end identification of a 3D rigid body dynamical system from visual observations. Adopting differentiable physics simulation, PIN-WM can be learned with only few-shot and task-agnostic physical interaction trajectories. Further, PIN-WM is learned with observational loss induced by Gaussian Splatting without needing state estimation. To bridge Sim2Real gaps, we turn the learned PIN-WM into a group of Digital Cousins via physics-aware randomizations which perturb physics and rendering parameters to generate diverse and meaningful variations of the PIN-WM. Extensive evaluations on both simulation and real-world tests demonstrate that PIN-WM, enhanced with physics-aware digital cousins, facilitates learning robust non-prehensile manipulation skills with Sim2Real transfer, surpassing the Real2Sim2Real state-of-the-arts.

Figures (11)

• Figure 1: PIN-WM is learned from few-shot and task-agnostic physical interaction trajectories (random pushes of the blocks in this example), through end-to-end differentiable identification of 3D physics parameters essential to the push operation (a). The learned PIN-WM is then turned into a group of digital cousins via physics-aware perturbations (b). The resulting world models are then used to learn the task-specific policies with Sim2Real transferability (c).
• Figure 2: Our Real2Sim2Real framework for learning non-prehensile manipulation policies. (a) The robot in the target domain moves around the object, capturing multi-view observations to estimate the rendering parameters $\bm{\alpha}$ of 2D Gaussian Splats. (b) Once optimized, $\bm{\alpha}$ is frozen. Both source and target domains apply the same task-agnostic physical interactions $\mathbf{a}_t$. In the source domain, dynamics are computed via LCP with physical parameters $\bm{\theta}$ to update the rendering. $\bm{\theta}$ is then optimized with the rendering loss between two domains. (c) The identified world model is then used for policy learning. Physics-aware perturbations are introduced to $\bm{\alpha}$ and $\bm{\theta}$ to mitigate the remained discrepancies from inaccurate observations. (d) This ensemble of perturbed world models enhances the Sim2Real transferability of learned policies.
• Figure 3: Manipulation trajectories in simulation obtained by our method for both push and flip tasks.
• Figure 4: Transition and orientation errors of push task during training.
• Figure 5: Our real-world experiment setup.