Neural Modes: Self-supervised Learning of Nonlinear Modal Subspaces

TL;DR

We address real-time physics-based simulation by learning nonlinear modal subspaces without curated data. The method optimizes a neural map to equilibrium configurations through energy minimization, formalized as $x^{*}(oldsymbol{Cphi},oldsymbol{Cpsi})= ext{argmin}_{x} E_{oldsymbol{Cphi}}(x)$ subject to $C_{oldsymbol{Cpsi}}(x)=0$, with a loss $L( heta)=\mathbb{E}_{oldsymbol{Cphi},oldsymbol{Cpsi}}[E_{oldsymbol{Cphi}}(x[\theta](oldsymbol{Cphi},oldsymbol{Cpsi}))+
0^a\|C_{oldsymbol{Cpsi}}(x[\theta](oldsymbol{Cphi},oldsymbol{Cpsi}))\|^2]$ and an origin penalty $Ceta\|y[\theta](0)\|^2$. This yields Neural Modes, a fully self-supervised, physically principled nonlinear subspace that outperforms geometry-based, supervised baselines, exhibits smooth and interpretable latent structure, avoids mode collapse, and enables real-time subspace dynamics and keyframing for deformable objects. The approach extends nonlinear compliant modes to multi-dimensional modal spaces and demonstrates real-time performance on shells and solids while maintaining physical fidelity. Its impact lies in enabling physically accurate, data-free real-time simulation and animation with interpretable, well-behaved latent representations.

Abstract

We propose a self-supervised approach for learning physics-based subspaces for real-time simulation. Existing learning-based methods construct subspaces by approximating pre-defined simulation data in a purely geometric way. However, this approach tends to produce high-energy configurations, leads to entangled latent space dimensions, and generalizes poorly beyond the training set. To overcome these limitations, we propose a self-supervised approach that directly minimizes the system's mechanical energy during training. We show that our method leads to learned subspaces that reflect physical equilibrium constraints, resolve overfitting issues of previous methods, and offer interpretable latent space parameters.

Figures (9)

• Figure 1: Visualization of our nonlinear subspace on a giraffe model discretized with tetrahedron finite elements. Overlays correspond to different frames from a selected neural mode.
• Figure 2: Supervised learning leads to overfitting for all loss functions, i.e., geometry-based L2 distance, average energy $E_{avg}$, and average stress $|S|_{avg}$. In contrast, our Neural Modes do not suffer from overfitting.
• Figure 3: Visualization of latent space dimensions for Neural Modes and PCA+AE.
• Figure 4: Correlation matrices of latent directions for Armadillo.
• Figure 5: Comparison of 5d Neural Modes, modal analysis, and full-space simulation on an Armadillo puppet.