ELITE: Efficient Gaussian Head Avatar from a Monocular Video via Learned Initialization and TEst-time Generative Adaptation

TL;DR

ELITE tackles high-fidelity monocular avatar synthesis by marrying a learned 3D prior (MGPM) with test-time 2D generative supervision, anchored by a rendering-guided single-step diffusion enhancer. The MGPM provides fast, identity-preserving initialization of a $2D$ Gaussian avatar from FLAME UV maps and driving signals, while Stage 1 real-image adaptation refines this prior to the target identity. Stage 2 leverages enhanced renderings from a single-step diffusion model to generate additional supervision and further adapt the avatar, yielding a final avatar that generalizes across unseen views and expressions with strong identity preservation. Compared to prior methods, ELITE achieves superior fidelity and dramatically faster synthesis (about $0.3$ seconds per image) than diffusion-based baselines, enabling practical, in-the-wild monocular avatar deployment.

Abstract

We introduce ELITE, an Efficient Gaussian head avatar synthesis from a monocular video via Learned Initialization and TEst-time generative adaptation. Prior works rely either on a 3D data prior or a 2D generative prior to compensate for missing visual cues in monocular videos. However, 3D data prior methods often struggle to generalize in-the-wild, while 2D generative prior methods are computationally heavy and prone to identity hallucination. We identify a complementary synergy between these two priors and design an efficient system that achieves high-fidelity animatable avatar synthesis with strong in-the-wild generalization. Specifically, we introduce a feed-forward Mesh2Gaussian Prior Model (MGPM) that enables fast initialization of a Gaussian avatar. To further bridge the domain gap at test time, we design a test-time generative adaptation stage, leveraging both real and synthetic images as supervision. Unlike previous full diffusion denoising strategies that are slow and hallucination-prone, we propose a rendering-guided single-step diffusion enhancer that restores missing visual details, grounded on Gaussian avatar renderings. Our experiments demonstrate that ELITE produces visually superior avatars to prior works, even for challenging expressions, while achieving 60x faster synthesis than the 2D generative prior method.

Figures (18)

• Figure 1: ELITE synthesizes an animatable photorealistic Gaussian head avatar from a casual monocular video. To compensate for missing views and expressions from the input video, ELITE leverages two complementary priors: (1) 3D data prior for feed-forward Gaussian initialization, and (2) 2D generative prior for augmenting unseen views and expressions for test-time adaptation. Compared to existing methods shao2024splattingavatartaubner2025cap4d that utilize no priors or only a 2D generative prior, ELITE achieves superior generalization across unseen views and expressions in the wild. Please refer to the supplementary video for dynamic avatar animation results.
• Figure 2: Comparison of existing avatar synthesis approaches. (a) Overfitting methods zielonka2023instashao2024splattingavatar optimize avatars from scratch, starting from 3D primitives anchored on a template mesh, and use only the input video frames as supervision. (b) 3D data prior methods zielonka2025synshotbuehler2024cafca use learned avatar initialization, but use only the input video frames as supervision. (c) 2D generative prior methods tang2025gaftaubner2025cap4d use diffusion-generated (full denoising, i.e., slow) images as test-time supervision, but optimize avatars from scratch. (d) Our ELITE enjoys the benefits of (b) and (c), i.e., we use learned avatar initialization and generated images as test-time supervision. We also generate images using a single-step diffusion that enhances Gaussian avatar renderings, significantly faster than full denoising methods tang2025gaftaubner2025cap4d.
• Figure 3: Training Mesh2Gaussian Prior Model (MGPM). We train a 3D avatar prior model, MGPM, that takes mesh UV maps and 3D face driving signals, e.g., expression codes, poses (jaw, eyes, neck, head), as inputs and outputs a Gaussian avatar, structured in the form of UV-aligned 2D Gaussian primitives. We supervise the MGPM training using images from the face capture dataset kirschstein2023nersemble that spans diverse identities across different expressions and viewpoints.
• Figure 4: Why need test-time avatar adaptation? (a) Our learned Gaussian initialization provides a visually reasonable initial, but synthesizing a high-fidelity avatar from only a feed-forward path is challenging at test time. (b) After the test-time adaptation of the avatar prior model, we obtain a high-fidelity, authentic avatar.
• Figure 5: Stage 1: Test-time adaptation w/ real images. Given input video frames and offline-tracked head mesh UV maps, we obtain 2D Gaussian UV maps by Mesh2Gaussian Prior Model's (MGPM) feed-forward avatar initialization. We fine-tune MGPM by minimizing the rendering loss between the animated Gaussian avatar images and the sampled image frames within the input video.