STMPL: Human Soft-Tissue Simulation

TL;DR

STMPL addresses the challenge of real-time soft-tissue deformation during object interactions by augmenting SMPL with a thick soft-tissue layer and a UV-map representation, enabling a 2D UNET to predict 3D deformations without runtime FEM. The approach learns from FEM-ground-truth data projected into 2D UV maps, producing fast, accurate deformation predictions that generalize to unseen object shapes and tissue thicknesses. Key contributions include a novel UV-based STMPL representation, a dataset of large-scale FEM-derived soft-tissue deformations, and a detailed evaluation showing strong accuracy and substantial speed-ups over traditional FEM. The method has practical impact for VR/AR and related fields by delivering plausible, interactive soft-tissue behavior with real-time performance, and it points to future extensions toward full-body nonuniform thickness and alternative learnable architectures.

Abstract

In various applications, such as virtual reality and gaming, simulating the deformation of soft tissues in the human body during interactions with external objects is essential. Traditionally, Finite Element Methods (FEM) have been employed for this purpose, but they tend to be slow and resource-intensive. In this paper, we propose a unified representation of human body shape and soft tissue with a data-driven simulator of non-rigid deformations. This approach enables rapid simulation of realistic interactions. Our method builds upon the SMPL model, which generates human body shapes considering rigid transformations. We extend SMPL by incorporating a soft tissue layer and an intuitive representation of external forces applied to the body during object interactions. Specifically, we mapped the 3D body shape and soft tissue and applied external forces to 2D UV maps. Leveraging a UNET architecture designed for 2D data, our approach achieves high-accuracy inference in real time. Our experiment shows that our method achieves plausible deformation of the soft tissue layer, even for unseen scenarios.

Figures (10)

• Figure 1: STMPL. STMPL's ability to simulate the soft tissue deformation of human bodies in real time is demonstrated through interactions with external interactions of varying shapes. We illustrate the capabilities of the STMPL model when simulating contact with grasping hands and 'ECCV' block letters, showcasing the detailed and realistic deformation such interactions produce.
• Figure 2: Method Overview. STMPL is based on a mapping between 3D (top) to 2D (bottom) of the body shape, the soft tissue layer, and the interacting external force. The generation of training data is conducted utilizing a 3D FEM simulator. The training process (bottom) is executed in a 2D framework, where input parameters such as body shape, soft tissue thickness, and external force are provided, while the FEM-derived deformation, projected onto a 2D space, serves as the ground truth for the output.
• Figure 3: Inference Pipeline. Inference is performed in 2D and takes as input interactions with complex shapes with variability of soft tissue thickness. The generated 2D deformation is mapped back into 3D to obtain the final deformed mesh.
• Figure 4: Soft Layer Generation. Top: five arms generated by the SMPL model with $\beta_{2}$ = [-2,-1,0,1,2]. Bottom: five arms with a range of thickness [0.5,0.625,0.75, 0.875, 1.0] cm.
• Figure 5: Training Data Span and Examples\ref{['fig:a']} and \ref{['fig:b']} illustrate the span of circular and elliptical force area of training data, \ref{['fig:c']} illustrates an example of seam force area, \ref{['fig:d']} shows a specific example of interaction with a disk, and \ref{['fig:e']} shows the vertices activated in simulation to create deformed result.