Transformer-based Neuro-Animator for Qualitative Simulation of Soft Body Movement

TL;DR

The paper addresses qualitative prediction of soft-body motion by introducing a visual transformer-based neuro-animator that predicts the next-frame 3D positions $P^{t+1}$ from the past sequence $[P^{t-n},...,P^t]$. It treats 3D particle trajectories from an 11-by-11 flag grid as tokens, embedding them into a trajectory-centric representation processed by a transformer with eight attention heads across eight layers, and trained with a robust Huber loss. The training data are generated from a mass-spring cloth simulation under gravity and three wind strengths, yielding approximately 15,000 sequences of length 64 across 121 trajectories. Results show learned temporal embeddings and plausible flag-waving under wind, though there remains room to improve motion naturalness and realism. This work demonstrates a memory-driven, qualitative visualization approach that can approximate dynamic physics without explicit numerical simulations, with potential applications in rapid qualitative forecasting of soft-body motion.

Abstract

The human mind effortlessly simulates the movements of objects governed by the laws of physics, such as a fluttering, or a waving flag under wind force, without understanding the underlying physics. This suggests that human cognition can predict the unfolding of physical events using an intuitive prediction process. This process might result from memory recall, yielding a qualitatively believable mental image, though it may not be exactly according to real-world physics. Drawing inspiration from the intriguing human ability to qualitatively visualize and describe dynamic events from past experiences without explicitly engaging in mathematical computations, this paper investigates the application of recent transformer architectures as a neuro-animator model. The visual transformer model is trained to predict flag motions at the \emph{t+1} time step, given information of previous motions from \emph{t-n} $\cdots$ \emph{t} time steps. The results show that the visual transformer-based architecture successfully learns temporal embedding of flag motions and produces reasonable quality simulations of flag waving under different wind forces.

Figures (3)

• Figure 1: Segmenting a flag to 10 rows and 10 columns results in 121 intersection points. $P_{i,j}^t$ represents an intersection point (i.e., a particle) at row $i$ and column $j$ at time step $t$. A model can learn to map a sequence of snap shots to the next snap shot: $\forall P_{i,j} \;\;\; [P_{i,j}^{t-n},...,P_{i,j}^{t}] \mapsto P_{i,j}^{t+1}$
• Figure 2: The transformer regressor model predicts the next positions of 121 points in the flag. Eight multi-head attention mechanisms were employed in each transformer layer, and the model implemented eight transformer layers. (see Alg. 1).
• Figure 3: Top pane: Training loss and validation loss from three neuro-animator models. Middle pane: Snapshots of predicted flag motion under three different wind strengths, from top to bottom: fluttering under strong wind, rippling under moderate wind, and hanging with occasional motions under light wind. Bottom pane: Each of these 121 plots displays a particle $(x, y, z)$ positions over 1,000 frames. In each miniature plot, x-axis representing 1000 time steps and y-axis represent the particles' positions $(x, y, z)$ normalized to [-1, 1], $x$ is red, $y$ is green and $z$ is blue.