SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatars

TL;DR

SurFhead tackles geometric inaccuracy in head avatars by moving beyond similarity transforms to affine deformations of 2D Gaussian surfels, governed by a Jacobian deformation gradient. A key innovation, Jacobian Blend Skinning (JBS), uses polar decomposition to smoothly interpolate rotation and stretch/shear across neighboring surfels, enabling stable extrapolation to unseen poses. The approach couples 2D Gaussian Splatting and mesh-binding (GaussianAvatars) with eyeball-aware regularization and anisotropic spherical Gaussians to preserve corneal convexity and specular highlights, resulting in high-fidelity geometry and appearance from RGB videos. Extensive experiments on synthetic and real datasets demonstrate superior geometry quality, robust reenactment, and competitive rendering speed, highlighting SurFhead’s potential for efficient, geometry-aware head avatars and relighting.

Abstract

Recent advancements in head avatar rendering using Gaussian primitives have achieved significantly high-fidelity results. Although precise head geometry is crucial for applications like mesh reconstruction and relighting, current methods struggle to capture intricate geometric details and render unseen poses due to their reliance on similarity transformations, which cannot handle stretch and shear transforms essential for detailed deformations of geometry. To address this, we propose SurFhead, a novel method that reconstructs riggable head geometry from RGB videos using 2D Gaussian surfels, which offer well-defined geometric properties, such as precise depth from fixed ray intersections and normals derived from their surface orientation, making them advantageous over 3D counterparts. SurFhead ensures high-fidelity rendering of both normals and images, even in extreme poses, by leveraging classical mesh-based deformation transfer and affine transformation interpolation. SurFhead introduces precise geometric deformation and blends surfels through polar decomposition of transformations, including those affecting normals. Our key contribution lies in bridging classical graphics techniques, such as mesh-based deformation, with modern Gaussian primitives, achieving state-of-the-art geometry reconstruction and rendering quality. Unlike previous avatar rendering approaches, SurFhead enables efficient reconstruction driven by Gaussian primitives while preserving high-fidelity geometry.

Figures (16)

• Figure 1: SurFhead reconstructs photo-realistic head avatars and high-fidelity surface normals, depth, and meshes from RGB videos alone. These avatars are represented through affine rigging of 2D surfel splats bound to a parametric morphable face model. SurFhead can fully control poses, expressions, and viewpoints, enhancing both appearance and geometry.
• Figure 2: Toy examples on Jacobian deformation and Polar Decomposition.
• Figure 3: Overall pipeline of SurFhead. Only from RGB videos, SurFhead constructs geometrically accurate head avatars, equipped with our intricate deformations. The Jacobian $\mathbf{J}$ covers stretch and shear deformations avoiding surface distortion. Moreover, the blended Jacobian $\mathbf{J}_\text{b}$ alleviates inherent local deformations' discontinuity. Finally, elaborated modeling of eyeballs such as preservation of specularity and convexity achieves more realistic appearance and geometry.
• Figure 4: Qualitative results on the FaceTalk dataset imavatar. Distinguished from other baselines, SurFhead simultaneously achieves convex eyeballs, and highly-detailed ears and nasal line. Best viewed when zoomed in.
• Figure 5: Qualitative results on NeRSemble dataset nersemble. Thanks to Jacobian and their blending, our method produces high-quality geometry with intricate details, visible in the normal maps. Please be aware of red boxes.