DEL: Discrete Element Learner for Learning 3D Particle Dynamics with Neural Rendering

TL;DR

The paper tackles learning 3D particle dynamics from 2D observations, where access to 3D ground-truth is limited. It introduces the Discrete Element Learner (DEL), which injects physical priors by replacing select DEA operators with learnable graph kernels, thereby constraining the learning problem to physics-guided operators rather than the entire dynamics. Training proceeds end-to-end with differentiable neural rendering, using pixel-based losses and a gradient constraint to align predicted motion with observed images. Experiments on a synthetic, multi-material dataset demonstrate that DEL outperforms state-of-the-art baselines, is robust to different renderers and camera counts, and supports material swapping, highlighting improved generalization and interpretability for 3D particle dynamics from 2D data.

Abstract

Learning-based simulators show great potential for simulating particle dynamics when 3D groundtruth is available, but per-particle correspondences are not always accessible. The development of neural rendering presents a new solution to this field to learn 3D dynamics from 2D images by inverse rendering. However, existing approaches still suffer from ill-posed natures resulting from the 2D to 3D uncertainty, for example, specific 2D images can correspond with various 3D particle distributions. To mitigate such uncertainty, we consider a conventional, mechanically interpretable framework as the physical priors and extend it to a learning-based version. In brief, we incorporate the learnable graph kernels into the classic Discrete Element Analysis (DEA) framework to implement a novel mechanics-integrated learning system. In this case, the graph network kernels are only used for approximating some specific mechanical operators in the DEA framework rather than the whole dynamics mapping. By integrating the strong physics priors, our methods can effectively learn the dynamics of various materials from the partial 2D observations in a unified manner. Experiments show that our approach outperforms other learned simulators by a large margin in this context and is robust to different renderers, fewer training samples, and fewer camera views.

Figures (30)

• Figure 1: The paradigm of the dynamics learning via inverse rendering. (a) Particles Initialization Process. The scene is initialized as particles. (b)Recurrent Dynamic Inference Process. The generated particle set is fed into a dynamic predictor to infer the next state iteratively.
• Figure 2: Two cases of particle interactions. (a) contact forces, affected by the intrusion $\delta d_n$. (b) The bond force exists between two particles of the same object, affected by the bond length.
• Figure 3: The main pipeline of message-passing network
• Figure 4: Qualitative Comparisons of dynamics prediction between our DEL and baselines in the particle-view on test sequences.
• Figure 5: Qualitative Comparisons of rendered images between ours and baselines.