GroomCap: High-Fidelity Prior-Free Hair Capture

TL;DR

GroomCap is introduced, a novel multi-view hair capture method that reconstructs faithful and high-fidelity hair geometry without relying on external data priors and proposes a Gaussian-based hair optimization strategy to refine the traced hair strands with a novel chained Gaussian representation.

Abstract

Despite recent advances in multi-view hair reconstruction, achieving strand-level precision remains a significant challenge due to inherent limitations in existing capture pipelines. We introduce GroomCap, a novel multi-view hair capture method that reconstructs faithful and high-fidelity hair geometry without relying on external data priors. To address the limitations of conventional reconstruction algorithms, we propose a neural implicit representation for hair volume that encodes high-resolution 3D orientation and occupancy from input views. This implicit hair volume is trained with a new volumetric 3D orientation rendering algorithm, coupled with 2D orientation distribution supervision, to effectively prevent the loss of structural information caused by undesired orientation blending. We further propose a Gaussian-based hair optimization strategy to refine the traced hair strands with a novel chained Gaussian representation, utilizing direct photometric supervision from images. Our results demonstrate that GroomCap is able to capture high-quality hair geometries that are not only more precise and detailed than existing methods but also versatile enough for a range of applications.

Figures (21)

• Figure 1: The input to our pipeline includes calibrated multi-view images (left), semantic segmentations of hair and foreground (middle), reconstructed inner and outer meshes with the hair bounding box (right), and optional hair partline annotation on one image (middle column, first row).
• Figure 2: The implicit hair volume network comprises three sub-modules: the feature network and appearance network are used to estimate view-independent volume density $\sigma$ and view-dependent radiance $\rva$ from input position $\rvx$ and view direction $\rvz$, similar to NeRF; an additional structure network is devised to estimate hair $\rho_h$ and body occupancy $\rho_b$ as well as 3D orientation $(\theta, \phi)$ in polar angles.
• Figure 3: Visualization of 3D orientation rendering and projection. We take two exemplary samples $t_1$ and $t_2$, characterized by 3D orientations of $(0.1\pi, 0.2\pi)$ and $(0.7\pi, 0.9\pi)$ in polar angles. Both samples reside on the same ray with transmittances of $0.6$ and $1.0$, where $t_1$ is closer to the camera. In the top row, from left to right, we show the expanded 3D orientation distributions of $t_1$, $t_2$, and their blended integration. In the bottom row, we illustrate the 2D orientation distribution after projecting their integrated 3D orientations using an exemplary camera matrix, detailed in \ref{['eq:proj']}.
• Figure 4: Visualization of orientation distributions. We start with an input image on the left, where we apply orientation filters and visualize the maximum responses in the middle. On the right, we select two exemplary patches (outlined by green and red rectangles on the left-hand side of the respective row), where the inner rectangles' sizes equal to the kernel radius. On the right-hand side, we illustrate the orientational distribution for each patch. For the green patch in the top row, there is a single sharp peak, indicating that most strands share the same direction. In contrast, the red patch in the bottom row shows a strong peak near $135^{\circ}$ and a secondary, broader peak from $30^{\circ}$ to $70^{\circ}$. The higher peak represents the thicker hairs that fill the lower-left half of the patch, while the other peak corresponds to the strands entering from the top-right corner. Merely keeping the maximum responses (middle) will omit these critical structural details.
• Figure 5: Reconstruction results on diverse hairstyles from short hairs to long ponytails, where personal features such as fringe, hairline, and clusters are faithfully captured. We use the same predefined material to better show geometric details.