VoMP: Predicting Volumetric Mechanical Property Fields

TL;DR

VoMP presents a fast, feed-forward approach to predict volumetric mechanical property fields $(E,\nu,\rho)$ for any geometry that can be voxelized and rendered from multiple views. Central to VoMP are a 2D material latent space (MatVAE) learned from real-world triplets, and a Geometry Transformer that maps voxelized multi-view features to per-voxel latents decoded into plausible material properties. The method is trained with a large, automatically annotated dataset (MTD and GVM) augmented by a vision-language model, enabling cross-representation generalization across meshes, Gaussian splats, SDFs, and NeRFs. Experiments show substantial accuracy and speed improvements over prior art, with end-to-end simulations validating the realism of predicted material fields. This work enables simulation-ready materials for diverse 3D representations, facilitating more faithful and scalable physical modeling in engineering workflows.

Abstract

Physical simulation relies on spatially-varying mechanical properties, often laboriously hand-crafted. VoMP is a feed-forward method trained to predict Young's modulus ($E$), Poisson's ratio ($ν$), and density ($ρ$) throughout the volume of 3D objects, in any representation that can be rendered and voxelized. VoMP aggregates per-voxel multi-view features and passes them to our trained Geometry Transformer to predict per-voxel material latent codes. These latents reside on a manifold of physically plausible materials, which we learn from a real-world dataset, guaranteeing the validity of decoded per-voxel materials. To obtain object-level training data, we propose an annotation pipeline combining knowledge from segmented 3D datasets, material databases, and a vision-language model, along with a new benchmark. Experiments show that VoMP estimates accurate volumetric properties, far outperforming prior art in accuracy and speed.

Figures (42)

• Figure 1: Wall-clock comparisons and breakdown.
• Figure 2: Simulator differences when dropping a solid sphere with $(E, \nu, \rho) = (10^4 Pa, 0.3, 10^3 \operatorname{kg/m}^3)$ with XPBD xpbd and MPM mpm vs. more accurate FEM.
• Figure 3: VoMP Overview. For any input geometry, we aggregate multi-view DINOv2 features across its volumetric voxelization (§\ref{['sec:aggregating']}). A trained GeometryTransformer (§\ref{['sec:3dencdoer']}) predicts per-voxel material latents, decoded by MatVAE (§\ref{['sec:mat_latent_space']}) into mechanical properties ($E$, $\nu$, $\rho$).
• Figure 4: Training Data annotation leverages accurate 3D data labels together with a VLM.
• Figure 5: Simulation-ready physics materials of VoMP enable realistic simulations for meshes and splats.