CAG-Avatar: Cross-Attention Guided Gaussian Avatars for High-Fidelity Head Reconstruction

TL;DR

Conditionally-Adaptive Gaussian Avatars (CAG-Avatar), a framework that resolves this key limitation of current animation techniques, is introduced, which confirms a significant improvement in reconstruction fidelity, particularly for challenging regions such as teeth, while preserving real-time rendering performance.

Abstract

Creating high-fidelity, real-time drivable 3D head avatars is a core challenge in digital animation. While 3D Gaussian Splashing (3D-GS) offers unprecedented rendering speed and quality, current animation techniques often rely on a "one-size-fits-all" global tuning approach, where all Gaussian primitives are uniformly driven by a single expression code. This simplistic approach fails to unravel the distinct dynamics of different facial regions, such as deformable skin versus rigid teeth, leading to significant blurring and distortion artifacts. We introduce Conditionally-Adaptive Gaussian Avatars (CAG-Avatar), a framework that resolves this key limitation. At its core is a Conditionally Adaptive Fusion Module built on cross-attention. This mechanism empowers each 3D Gaussian to act as a query, adaptively extracting relevant driving signals from the global expression code based on its canonical position. This "tailor-made" conditioning strategy drastically enhances the modeling of fine-grained, localized dynamics. Our experiments confirm a significant improvement in reconstruction fidelity, particularly for challenging regions such as teeth, while preserving real-time rendering performance.

Figures (4)

• Figure 1: Diagram of the proposed method. We represent the 3D head as a Gaussian field initialized in a 2D UV space and subsequently embedded onto a dynamic FLAME mesh surface via mesh rasterization. To animate the Gaussians, the canonical position of each Gaussian center is first encoded into a positional feature vector. This feature vector is then refined using the tracked expression code through a cross-attention fusion module, where positional features act as queries and the global expression code provides the key and value. The resulting expression-conditioned features are integrated with the original positional features via a residual connection. This fused representation is then fed into an offset MLP to predict the final spatial deformation, including offsets in position, rotation, and scale. Finally, the deformed Gaussians are splatted to render the portrait image under a given camera pose.
• Figure 2: Qualitative comparison with state-of-the-art head portrait reconstruction methods. Our model demonstrates high fidelity in reconstructing fine facial details and subtle expressions. Notably, through our proposed cross-attention fusion mechanism, our method achieves significantly improved reconstruction of challenging non-skin regions, such as teeth, compared to prior works.
• Figure 3: Qualitative results of our method on the face reenactment task. Our method preserves personalized facial details in the hair, eye, and mouth regions and synthesizes more natural results.
• Figure 4: Demonstration of new 3D consistent head pose synthesis and expression variations.