HeadLighter: Disentangling Illumination in Generative 3D Gaussian Heads via Lightstage Captures

TL;DR

HeadLighter tackles the challenge of disentangling illumination from intrinsic head appearance in 3D Gaussian head models. It introduces a dual-branch generator that separates geometry/albedo from relighting attributes and trains them progressively with real-world lightstage captures, enabling physically based rendering under arbitrary lighting. A normal-distillation strategy and joint end-to-end training yield photorealistic relighting, explicit lighting control, and real-time rendering while preserving identity. Built on a lightstage dataset and a three-stage workflow, HeadLighter demonstrates high-fidelity intrinsic attribute synthesis, robust lighting manipulation, and material editing, with practical implications for digital avatars and XR applications.

Abstract

Recent 3D-aware head generative models based on 3D Gaussian Splatting achieve real-time, photorealistic and view-consistent head synthesis. However, a fundamental limitation persists: the deep entanglement of illumination and intrinsic appearance prevents controllable relighting. Existing disentanglement methods rely on strong assumptions to enable weakly supervised learning, which restricts their capacity for complex illumination. To address this challenge, we introduce HeadLighter, a novel supervised framework that learns a physically plausible decomposition of appearance and illumination in head generative models. Specifically, we design a dual-branch architecture that separately models lighting-invariant head attributes and physically grounded rendering components. A progressive disentanglement training is employed to gradually inject head appearance priors into the generative architecture, supervised by multi-view images captured under controlled light conditions with a light stage setup. We further introduce a distillation strategy to generate high-quality normals for realistic rendering. Experiments demonstrate that our method preserves high-quality generation and real-time rendering, while simultaneously supporting explicit lighting and viewpoint editing. We will publicly release our code and dataset.

Figures (10)

• Figure 1: Our proposed framework. We leverage real-world captures to learn high-quality disentanglement of illumination and appearance in generative 3D gaussian heads. We capture multi-view images of different subjects under various light conditions via lightstage. A dual-branch architecture is employed to separately generate light-invariant gaussian attributes (geometry $\mathcal{A}_{geo}$ and albedo $\rho$), and physically-based rendering attributes (normal $\mathbf{n}_k$, roughness$\sigma_k$ and diffuse light transport coefficients $T_k$). These attributes together with camera pose and light condition can be splatted into images and compared to the real-world captures to optimize the network.
• Figure 2: 3D head generation and lighting control. EG3D and GGHEAD do not support explicit lighting control. NFL and GSHR support lighting editing but struggle to handle extreme input lighting (strong, side light), while our method can effectively interact with such challenging conditions. Lighting conditions are provided at the lower right corner.
• Figure 3: Relighting under complex environment maps. NFL assumes white illumination and fails to handle colored lighting. GSHR lacks real-world OLAT supervision and therefore cannot produce physically plausible responses under complex lighting. In contrast, our method, trained with multi-view OLAT captures, accurately renders realistic shading and reflections under diverse and challenging illumination conditions.
• Figure 4: Free view relighting on captured lightstage data. We show the lighting condition at the lower right corner.
• Figure 5: Normal generation with different methods.