A Physics-informed Demonstration-guided Learning Framework for Granular Material Manipulation

TL;DR

The paper tackles the challenging problem of robot manipulation of granular materials by introducing a physics-informed learning framework that couples a differentiable MLS-MPM-based simulator with a Drucker–Prager yield model, an automatic demonstration generator, and a demonstration-guided reinforcement learning module (DG-SAC). By generating gradient-based demonstrations from fluid-like or elasto-plastic models and then training a robust RL policy with skill chaining and enhanced rewards, the approach overcomes gradient instability and data-efficiency issues common in granular manipulation. Empirical results in both simulation and real-world kitchens show DG-SAC outperforms standard RL/IL baselines and a granular GNN baseline, while maintaining robustness across varying material properties and enabling sim-to-real transfer. The framework offers a scalable path to long-horizon, contact-rich manipulation tasks without extensive real-world data collection, with practical implications for household and industrial granular handling.

Abstract

Due to the complex physical properties of granular materials, research on robot learning for manipulating such materials predominantly either disregards the consideration of their physical characteristics or uses surrogate models to approximate their physical properties. Learning to manipulate granular materials based on physical information obtained through precise modelling remains an unsolved problem. In this paper, we propose to address this challenge by constructing a differentiable physics-based simulator for granular materials using the Taichi programming language and developing a learning framework accelerated by demonstrations generated through gradient-based optimisation on non-granular materials within our simulator, eliminating the costly data collection and model training of prior methods. Experimental results show that our method, with its flexible design, trains robust policies that are capable of executing the task of transporting granular materials in both simulated and real-world environments, beyond the capabilities of standard reinforcement learning, imitation learning, and prior task-specific granular manipulation methods.

Figures (6)

• Figure 1: Granular material manipulation in our simulator (above) and real environment (below) for one representative task, where the agent uses a spoon to follow the optimised trajectory and completes scooping, translating, and pouring sub-tasks.
• Figure 2: Illustration of the problem setting. In our study, four tasks are proposed: transporting granular materials using a spoon (blue box), a scoop (green box), a bottle (red box), and a shovel (purple box), respectively. The tasks involving the spoon and scoop consist of three sub-tasks: scooping, translating, and pouring. For each action, image 1 denotes the initial state, while image 2 denotes the state along the optimal trajectory trained by our model.
• Figure 3: Workflow of the proposed learning framework. Green arrows: imperfect demonstrations generated via gradient-based optimisation with a fluid or elasto-plastic material model. Brown arrows: SAC training with dual replay buffers — a fixed buffer storing the demonstrations and an updated buffer collecting data from interaction with the actual granular dynamics. The final output is an $N_a*d_a$-dimensional policy over $N_H$ time steps.
• Figure 4: We employ the concept of skill chaining, innovatively integrating an Euler angle objective function $\mathcal{J}_{s}$ within the learning paradigm of the scooping sub-tasks. This function is designed to drive the agent towards achieving a seamless connection between scooping and translating actions.
• Figure 5: The variation in gradients at different timesteps during the backpropagation phase for fluids or elasto-plastic materials (blue) and granular materials (orange) in the first iteration of trajectory optimisation across different sub-tasks.