SVAD: From Single Image to 3D Avatar via Synthetic Data Generation with Video Diffusion and Data Augmentation

TL;DR

SVAD tackles animatable 3D-avatar generation from a single image by generating synthetic, pose-conditioned training data with a video diffusion model (MusePose) and refining it via identity-preservation and image restoration before training a 3D Gaussian Splatting (3DGS) avatar. The method combines diffusion’s generative flexibility with 3DGS’ rendering efficiency to maintain identity across novel poses and viewpoints while enabling real-time rendering. Key contributions include a full data-augmentation pipeline that preserves facial identity, a diffusion-guided data synthesis process, and a SMPL-X–driven 3DGS training regime with Laplacian regularization and hierarchical SH growth, achieving state-of-the-art performance on single-image avatar tasks. The approach generalizes to standard datasets (People Snapshot, THuman) and supports extensions like text-to-avatar and text-guided editing, offering practical impact for VR/AR and digital entertainment while acknowledging limitations in background segmentation, clothing variability, and computational demands ($L = \lambda_{RGB}L_{RGB} + \lambda_{SSIM}L_{SSIM} + \lambda_{LPIPS}L_{LPIPS}$).

Abstract

Creating high-quality animatable 3D human avatars from a single image remains a significant challenge in computer vision due to the inherent difficulty of reconstructing complete 3D information from a single viewpoint. Current approaches face a clear limitation: 3D Gaussian Splatting (3DGS) methods produce high-quality results but require multiple views or video sequences, while video diffusion models can generate animations from single images but struggle with consistency and identity preservation. We present SVAD, a novel approach that addresses these limitations by leveraging complementary strengths of existing techniques. Our method generates synthetic training data through video diffusion, enhances it with identity preservation and image restoration modules, and utilizes this refined data to train 3DGS avatars. Comprehensive evaluations demonstrate that SVAD outperforms state-of-the-art (SOTA) single-image methods in maintaining identity consistency and fine details across novel poses and viewpoints, while enabling real-time rendering capabilities. Through our data augmentation pipeline, we overcome the dependency on dense monocular or multi-view training data typically required by traditional 3DGS approaches. Extensive quantitative, qualitative comparisons show our method achieves superior performance across multiple metrics against baseline models. By effectively combining the generative power of diffusion models with both the high-quality results and rendering efficiency of 3DGS, our work establishes a new approach for high-fidelity avatar generation from a single image input.

Figures (18)

• Figure 1: SVAD. Our method creates high-fidelity 3D avatars from a single image through synthetic data generation. We leverage video diffusion to generate pose-conditioned animations, enhance them with identity preservation and image restoration modules, then train a 3D Gaussian Splatting avatar. The resulting avatars maintain consistent identity across novel poses and viewpoints while enabling real-time rendering, outperforming state-of-the-art approaches.
• Figure 2: Overall Pipeline of SVAD. Starting from a single input image, the diffusion model generates pose-conditioned animations, which are refined using an identity preservation module and an image restoration module. The refined outputs are then used to train the 3DGS avatar, enabling high-fidelity, animatable 3D avatars with consistent details across poses and viewpoints.
• Figure 3: 3D Avatars trained by SVAD. SVAD generates high quality 3D avatars with just a single-image. The trained avatars can be rendered from any view point, in any pose.
• Figure 4: Qualitative Evaluation on the People Snapshot dataset and of THuman dataset scan renderings. From a single-image input, SVAD generates high-quality, animatable 3D avatars.
• Figure 5: Qualitative Evaluation against SiTH ho2024sith. Our approach better reconstructs complex contours and subtle features, resulting in a more lifelike and coherent side-view appearance.