CaricatureGS: Exaggerating 3D Gaussian Splatting Faces With Gaussian Curvature

Abstract

A photorealistic and controllable 3D caricaturization framework for faces is introduced. We start with an intrinsic Gaussian curvature-based surface exaggeration technique, which, when coupled with texture, tends to produce over-smoothed renders. To address this, we resort to 3D Gaussian Splatting (3DGS), which has recently been shown to produce realistic free-viewpoint avatars. Given a multiview sequence, we extract a FLAME mesh, solve a curvature-weighted Poisson equation, and obtain its exaggerated form. However, directly deforming the Gaussians yields poor results, necessitating the synthesis of pseudo-ground-truth caricature images by warping each frame to its exaggerated 2D representation using local affine transformations. We then devise a training scheme that alternates real and synthesized supervision, enabling a single Gaussian collection to represent both natural and exaggerated avatars. This scheme improves fidelity, supports local edits, and allows continuous control over the intensity of the caricature. In order to achieve real-time deformations, an efficient interpolation between the original and exaggerated surfaces is introduced. We further analyze and show that it has a bounded deviation from closed-form solutions. In both quantitative and qualitative evaluations, our results outperform prior work, delivering photorealistic, geometry-controlled caricature avatars.

Figures (12)

• Figure 1: Photorealistic 3D caricature avatars produced by our method.
• Figure 2: CaricatureGS generation framework.(1) From a subject’s multi-view video, we extract a FLAME mesh and compute a curvature-driven caricature based on it. Combined with subject-specific FLAME parameters, this yields the subject’s caricature mesh. (2) Per-triangle 2D affine transforms map the neutral mesh projection to its caricatured counterpart, warping each frame to generate pseudo–ground-truth image pairs. (3) Anisotropic 3D Gaussians primitives are bound to the original mesh and transformed to the caricature mesh via the corresponding 3D triangle transforms. Rendered neutral and caricature views are alternated and compared to their pseudo–ground-truth counterparts in joint optimization.
• Figure 3: Parametric trend of the error with respect to $\gamma$. The error, normalized by the bounding-box diagonal of the mesh, increases from both ends of $\gamma$, reaching a negligible maximum at $\tfrac{\gamma_f}{2}$, where $\gamma_f=0.25$.
• Figure 4: Visualizations of localized, semantically controlled facial exaggerations.
• Figure 5: Rendering results from our pipeline lee_surfhead_2024. SURFHEAD: Caricature generation by first reconstructing an avatar with the state-of-the-art SURFHEAD model lee_surfhead_2024, followed by mesh exaggeration. Ours: Renderings across different caricature intensities. Our approximation-based control interpolates smoothly along the caricature intensity axis while preserving visual fidelity.