EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics

TL;DR

The paper tackles learning graph dynamics of cloth-like deformables with unknown elastic properties by introducing EDO-Net, which uses a pulling-based Exploratory Action to infer a latent property vector $z_i$ via an adaptation module $f_\phi$ and a forward dynamics module $g_\theta$ (a GNN) conditioned on $z_i$ to predict future graph states $\delta \hat{G}^i_t$. Trained on a diverse set of samples across a distribution $\mathcal{T}$ of elastic properties without ground-truth property labels, the model learns to generalize to unseen materials and transfers to downstream tasks such as inverse dynamics and cross-environment scenarios. Empirical results in simulation and real-world data show that decoding $z_i$ captures ground-truth physical parameters, that $z_i$ transfers across environments, and that conditioning the forward model on $z_i$ improves prediction accuracy over non-conditioned baselines. This work advances robust manipulation of cloth-like objects by enabling adaptive dynamics with latent physical-property representations, with potential to extend to multiple exploratory actions and broader manipulation tasks.

Abstract

We study the problem of learning graph dynamics of deformable objects that generalizes to unknown physical properties. Our key insight is to leverage a latent representation of elastic physical properties of cloth-like deformable objects that can be extracted, for example, from a pulling interaction. In this paper we propose EDO-Net (Elastic Deformable Object - Net), a model of graph dynamics trained on a large variety of samples with different elastic properties that does not rely on ground-truth labels of the properties. EDO-Net jointly learns an adaptation module, and a forward-dynamics module. The former is responsible for extracting a latent representation of the physical properties of the object, while the latter leverages the latent representation to predict future states of cloth-like objects represented as graphs. We evaluate EDO-Net both in simulation and real world, assessing its capabilities of: 1) generalizing to unknown physical properties, 2) transferring the learned representation to new downstream tasks.

Figures (7)

• Figure 1: A pulling interaction is leveraged by EDO-Net to explore the elastic properties of the object, which improves the performance in subsequent tasks such as partial bandage.
• Figure 2: Scheme of the overall model. Given a deformable object $\mathcal{T}_i$ with unknown physical properties, the adaptation module $f_\phi$ updates the initialization $z_0$ of the latent representation of the physical properties $\mathcal{T}_i$ from sequences of observations $O_t^{i}|_{t=1, ..., T}$ processed by an attention layer and a RNN. In a second phase, the forward dynamics module $g_\theta$, implemented as a gnn, uses $z_i$ obtained from the adaptation module to predict future states $\hat{G}_{t}$ of the deformable object.
• Figure 3: Pulling Exploratory Actions to observe graphs and forces.
• Figure 4: The environments employed to evaluate EDO-Net.
• Figure 5: Textile dataset samples (top) and procedure to extract graphs from point cloud (bottom). First we represent the grippers as 8 equidistant nodes (a). We then slice the point cloud with a plane passing through 2 corresponding nodes of the grippers, obtaining 6 additional equidistant nodes (b). We obtain the final graph by connecting the neighbors of each node as shown in (c).