MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics

TL;DR

MPMAvatar combines a tailored anisotropic Material Point Method simulation with a hybrid mesh-Gaussian representation to reconstruct physically accurate, robustly animated 3D avatars from multi-view videos. The approach explicitly models garment dynamics under complex contact with a body mesh and renders with high fidelity via 3D Gaussian Splats and quasi-shadowing, achieving state-of-the-art dynamics and rendering performance. A key strength is zero-shot generalization to unseen scene interactions, enabled by the physics priors and mesh-based collision handling. The method demonstrates substantial robustness and efficiency improvements over prior physics-based avatars, with potential impact on virtual reality, digital fashion, and content creation, while acknowledging limitations such as lack of relighting and opportunities for further physical realism enhancements.

Abstract

While there has been significant progress in the field of 3D avatar creation from visual observations, modeling physically plausible dynamics of humans with loose garments remains a challenging problem. Although a few existing works address this problem by leveraging physical simulation, they suffer from limited accuracy or robustness to novel animation inputs. In this work, we present MPMAvatar, a framework for creating 3D human avatars from multi-view videos that supports highly realistic, robust animation, as well as photorealistic rendering from free viewpoints. For accurate and robust dynamics modeling, our key idea is to use a Material Point Method-based simulator, which we carefully tailor to model garments with complex deformations and contact with the underlying body by incorporating an anisotropic constitutive model and a novel collision handling algorithm. We combine this dynamics modeling scheme with our canonical avatar that can be rendered using 3D Gaussian Splatting with quasi-shadowing, enabling high-fidelity rendering for physically realistic animations. In our experiments, we demonstrate that MPMAvatar significantly outperforms the existing state-of-the-art physics-based avatar in terms of (1) dynamics modeling accuracy, (2) rendering accuracy, and (3) robustness and efficiency. Additionally, we present a novel application in which our avatar generalizes to unseen interactions in a zero-shot manner-which was not achievable with previous learning-based methods due to their limited simulation generalizability. Our project page is at: https://KAISTChangmin.github.io/MPMAvatar/

Figures (4)

• Figure 1: Overview of our dynamic avatar modeling. We hybridly represent our canonical avatar with (1) a mesh with physical parameters for geometry and dynamics modeling, and (2) 3D Gaussian Splats kerbl20233d for appearance modeling (Sec. \ref{['subsec:avatar_representation']}). This avatar can be animated via linear blend skinning for non-garment regions and physical simulation for garment regions (Sec. \ref{['subsubsec:aniso']}) with our novel collision handling algorithm (Sec. \ref{['subsec:body_collider']}). Visualization key. Blue arrows indicate body grid velocities, green arrows denote garment grid velocities, and red arrows show colliding grid regions where velocity projection is applied.
• Figure 2: Qualitative results on test frames in the ActorsHQ icsik2023humanrf and 4D-DRESS wang20244d datasets. Our method outperforms PhysAvatar zheng2024physavatar in appearance by rendering sharper, less blurred textures with finer detail and in geometry by recovering folds and wrinkles that more closely match the ground truth.
• Figure 3: Qualitative ablation study results.
• Figure 4: Qualitative comparison against concurrent baselines on the ActorsHQ icsik2023humanrf dataset. We compare our method with two recent concurrent methods: Gaussian Garments rong2024gaussian and MMLPHuman zhan2025mmlphuman. Gaussian Garments rong2024gaussian struggles to produce physically accurate deformations, while MMLPHuman zhan2025mmlphuman exhibits unnatural surface artifacts or discontinuities under challenging poses. In contrast, our method yields more realistic and plausible garment dynamics and geometry.