Creating Your Editable 3D Photorealistic Avatar with Tetrahedron-constrained Gaussian Splatting

TL;DR

This work introduces TetGS, a hybrid tetrahedron-constrained Gaussian Splatting representation for editable 3D avatars, enabling precise region-specific geometry edits and photorealistic texture generation from monocular video. By decoupling editing into localized geometry adaptation and appearance learning, TetGS embeds Gaussians inside explicit tetrahedral grids and guides their movement via SDF updates, with a three-stage pipeline: high-fidelity avatar instantiation, localized spatial adaptation, and texture generation with progressive refinement. The approach supports text- and reference-image-guided editing, including 360° full-body avatars and garment transfer for 3D virtual try-on, and outperforms state-of-the-art 3DGS editors in both qualitative and quantitative evaluations. The method offers a practical, user-friendly path to realistic, editable digital humans for AR/VR and e-commerce applications, while also outlining limitations and directions for future improvements.

Abstract

Personalized 3D avatar editing holds significant promise due to its user-friendliness and availability to applications such as AR/VR and virtual try-ons. Previous studies have explored the feasibility of 3D editing, but often struggle to generate visually pleasing results, possibly due to the unstable representation learning under mixed optimization of geometry and texture in complicated reconstructed scenarios. In this paper, we aim to provide an accessible solution for ordinary users to create their editable 3D avatars with precise region localization, geometric adaptability, and photorealistic renderings. To tackle this challenge, we introduce a meticulously designed framework that decouples the editing process into local spatial adaptation and realistic appearance learning, utilizing a hybrid Tetrahedron-constrained Gaussian Splatting (TetGS) as the underlying representation. TetGS combines the controllable explicit structure of tetrahedral grids with the high-precision rendering capabilities of 3D Gaussian Splatting and is optimized in a progressive manner comprising three stages: 3D avatar instantiation from real-world monocular videos to provide accurate priors for TetGS initialization; localized spatial adaptation with explicitly partitioned tetrahedrons to guide the redistribution of Gaussian kernels; and geometry-based appearance generation with a coarse-to-fine activation strategy. Both qualitative and quantitative experiments demonstrate the effectiveness and superiority of our approach in generating photorealistic 3D editable avatars.

Figures (15)

• Figure 1: An overview of our method, built upon the proposed hybrid Tetrahedron-constrained Gaussian Splatting (TetGS). Our method first learns accurate TetGS initialization from a monocular video, then updates the spatial allocation of localized editing Gaussians along with explicitly partitioned tetrahedrons under diffusion guidance. With the learned distribution, we perform texture editing by optimizing restricted Gaussians with few-shot inpainted images and activating their attributes under augmented guidance.
• Figure 2: An illustration of tetrahedron-constrained Gaussian. Each Gaussian kernel is embedded in a unique tetrahedron with its position $\mu$ calculated from the SDF of tetrahedral vertices.
• Figure 3: Demonstration of the tetrahedron partitioning process. (i) Localized mesh triangles given multi-view mask labels. (ii) Tetrahedron partitioning according to their extracted triangles. (iii) Tetrahedral vertices grouping into frozen and editable ones.
• Figure 4: Multi-view renderings and the underlying geometries before and after editing with various subjects and accessories.
• Figure 5: Qualitative comparison with text-guided methods GaussianEditor wang2024gaussianeditor and DGE chen2024dge, and image-guided method TIP-Editor zhuang2024tip.