GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians

TL;DR

GaussianHair presents an explicit hair representation that models each strand as a sequence of cylindrical 3D Gaussian primitives, enabling accurate geometry and appearance reconstruction from handheld video with differentiable rendering. It couples this geometry proxy with a Marschner-inspired scattering model implemented via UE4 approximations to achieve photorealistic relighting and dynamic rendering while remaining compatible with standard CG pipelines. The method optimizes strand geometry from an Oriented 3D Gaussian Field and a geometry texture decoded by an MLP, trained with a suite of losses including photometric, alpha, and strand-smoothness terms, and supports efficient pruning and duplication to balance detail with performance. A new RealHair dataset of 281 high-resolution videos and meticulous strand annotations under uniform lighting underpins extensive qualitative and quantitative evaluation, showing superior geometry fidelity and competitive appearance relative to state-of-the-art methods, along with practical editing relighting and dynamic rendering capabilities that advance digital hair production and inclusivity in visual media.

Abstract

Hairstyle reflects culture and ethnicity at first glance. In the digital era, various realistic human hairstyles are also critical to high-fidelity digital human assets for beauty and inclusivity. Yet, realistic hair modeling and real-time rendering for animation is a formidable challenge due to its sheer number of strands, complicated structures of geometry, and sophisticated interaction with light. This paper presents GaussianHair, a novel explicit hair representation. It enables comprehensive modeling of hair geometry and appearance from images, fostering innovative illumination effects and dynamic animation capabilities. At the heart of GaussianHair is the novel concept of representing each hair strand as a sequence of connected cylindrical 3D Gaussian primitives. This approach not only retains the hair's geometric structure and appearance but also allows for efficient rasterization onto a 2D image plane, facilitating differentiable volumetric rendering. We further enhance this model with the "GaussianHair Scattering Model", adept at recreating the slender structure of hair strands and accurately capturing their local diffuse color in uniform lighting. Through extensive experiments, we substantiate that GaussianHair achieves breakthroughs in both geometric and appearance fidelity, transcending the limitations encountered in state-of-the-art methods for hair reconstruction. Beyond representation, GaussianHair extends to support editing, relighting, and dynamic rendering of hair, offering seamless integration with conventional CG pipeline workflows. Complementing these advancements, we have compiled an extensive dataset of real human hair, each with meticulously detailed strand geometry, to propel further research in this field.

Figures (15)

• Figure 1: Illustration of Our GaussianHair. We introduce "GaussianHair", a novel hair representation technique that conceptualizes a hair strand as a series of connected cylindrical 3D Gaussians. This representation facilitates the effective reconstruction of hair strands from videos captured using handheld smartphones, while also supporting an efficient scattering model. Leveraging the "GaussianHair" framework, image-based hair modeling extends beyond mere reconstruction, enabling advanced functionalities such as hair editing, relighting, and dynamic rendering.
• Figure 2: Illustration of our RealHair dataset. Our RealHair dataset represents a comprehensive and culturally diverse collection of human hairstyles, encompassing a variety of distinctive styles reflective of global hair characteristics. It comprises 281 high-resolution (4K) videos, totaling approximately 3000 frames, each meticulously annotated with detailed geometry segmentations and individual hair strand information.
• Figure 3: Overview. Our method employs a multi-stage process for hair modeling. Initially, an off-the-shelf decoder extracts coarse hair strands from multi-view images, which are then refined using differentiable strand-based splatting. This optimization aligns the rendered images with the ground truth. Finally, we apply a scattering model to the optimized strands, enhancing their relighting and dynamics modeling capabilities.
• Figure 4: Gaussian Hair Representation. A hair strand is represented as a sequence of linked cylindrical 3D Gaussian primitives with their length $s$ significantly larger than their diameter $r$. During the modeling process, initialized strands are optimized to the optimal structures.
• Figure 5: Illustration of coarse strands generation. Given fitted FLAME head mesh, we render a hair geometry texture in a differentiable manner with the 2D feature map as a UV map. Subsequently, an off-the-shelf decoder is utilized to obtain a set of hair strands which are then optimized to align with the actual hair geometry.