TexAvatars : Hybrid Texel-3D Representations for Stable Rigging of Photorealistic Gaussian Head Avatars

TL;DR

TexAvatars tackles the challenge of stable, photorealistic Gaussian head avatars by unifying mesh-grounded analytic rigging with texel-space neural regression. Local Gaussian attributes are regressed in UV space and lifted to 3D through a mesh-aware Quasi-Phong Jacobian Field, enabling smooth, semantically meaningful deformations across triangle boundaries. This decouples semantic modeling from geometric control, improving generalization and stability while capturing fine wrinkles and intraoral geometry. Evaluations on NeRSemble show state-of-the-art performance under extreme pose and expression variations with strong cross-identity and cross-expression generalization, while maintaining real-time rendering capability.

Abstract

Constructing drivable and photorealistic 3D head avatars has become a central task in AR/XR, enabling immersive and expressive user experiences. With the emergence of high-fidelity and efficient representations such as 3D Gaussians, recent works have pushed toward ultra-detailed head avatars. Existing approaches typically fall into two categories: rule-based analytic rigging or neural network-based deformation fields. While effective in constrained settings, both approaches often fail to generalize to unseen expressions and poses, particularly in extreme reenactment scenarios. Other methods constrain Gaussians to the global texel space of 3DMMs to reduce rendering complexity. However, these texel-based avatars tend to underutilize the underlying mesh structure. They apply minimal analytic deformation and rely heavily on neural regressors and heuristic regularization in UV space, which weakens geometric consistency and limits extrapolation to complex, out-of-distribution deformations. To address these limitations, we introduce TexAvatars, a hybrid avatar representation that combines the explicit geometric grounding of analytic rigging with the spatial continuity of texel space. Our approach predicts local geometric attributes in UV space via CNNs, but drives 3D deformation through mesh-aware Jacobians, enabling smooth and semantically meaningful transitions across triangle boundaries. This hybrid design separates semantic modeling from geometric control, resulting in improved generalization, interpretability, and stability. Furthermore, TexAvatars captures fine-grained expression effects, including muscle-induced wrinkles, glabellar lines, and realistic mouth cavity geometry, with high fidelity. Our method achieves state-of-the-art performance under extreme pose and expression variations, demonstrating strong generalization in challenging head reenactment settings.

Figures (10)

• Figure 1: Overview. These local attributes are lifted to global space by transforming them with precomputed Jacobians $\mathbf{J}_\text{uv}$, remapped from the tracked mesh to texel space by remapping function $\phi_F$. This results in globally coherent attributes that are continuous across surface regions via linear grid sampling.
• Figure 2: Extreme Self- and Cross-Reenactment Scenario. Our approach demonstrates significantly higher fidelity under extreme facial motions and rigid head rotations (subjects from NeRSemble nersemble, FREE corpus). Notably, our model accurately reconstructs high-frequency features such as nasolabial lines, hair strands, and detailed oral cavity structures. These results highlight the effectiveness of our hybrid texel-rigging framework in preserving semantic details even under highly expressive and challenging settings.
• Figure 3: Comparison with RGCA rgca. RGCA adopts a relatively large mesh scale to stabilize training by keeping offsets small, which works well in most cases but can still cause blobby artifacts such as blurred nasal lines under strong stretching.
• Figure 4: Effect of Image Animation Model. It enables the synthesis of details that are not explicitly represented in 3DMMs, such as wrinkles or subtle skin deformations, thereby enhancing realism under dynamic expressions.
• Figure 5: Effect of Global UV Sampling and Jacobian. Remapping mesh Jacobians to texel space enables smooth blending of attributes across triangle boundaries. Jacobian-based deformation effectively models stretch and anisotropic scaling while reducing blob-like artifacts.