BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting

TL;DR

BiGS introduces Bidirectional Gaussian Primitives to enable relighting and novel view synthesis for Gaussian Splatting under dynamic illumination. By modeling the per-primitive appearance with an intrinsic light decomposition $L(\omega_o) = L_{dir}(\omega_o) + L_{ind}$ and representing direct/diffuse components via $\mathcal{T}_{dir}$, $\mathcal{T}_{ind}$, and spherical harmonics, including a bidirectional scattering term $s(\omega_i, \omega_o)$ with reciprocity, BiGS unifies surface and volumetric appearance while remaining compatible with rasterization. The approach is trained end-to-end on OLAT data using reconstruction and regularization losses that enforce energy conservation and non-negativity, enabling physically plausible relighting under point lights, directional lights, and environment maps. Experimental results on synthetic and captured OLAT datasets demonstrate accurate relighting for diverse materials, including subsurface scattering in translucent objects, and show favorable comparisons to prior surface-based or implicit methods in preserving volumetric light transport. This work enables real-time, photorealistic rendering of complex materials under novel lighting, with broad implications for virtual production, AR/VR, and interactive graphics.

Abstract

We present Bidirectional Gaussian Primitives, an image-based novel view synthesis technique designed to represent and render 3D objects with surface and volumetric materials under dynamic illumination. Our approach integrates light intrinsic decomposition into the Gaussian splatting framework, enabling real-time relighting of 3D objects. To unify surface and volumetric material within a cohesive appearance model, we adopt a light- and view-dependent scattering representation via bidirectional spherical harmonics. Our model does not use a specific surface normal-related reflectance function, making it more compatible with volumetric representations like Gaussian splatting, where the normals are undefined. We demonstrate our method by reconstructing and rendering objects with complex materials. Using One-Light-At-a-Time (OLAT) data as input, we can reproduce photorealistic appearances under novel lighting conditions in real time.

Figures (11)

• Figure 1: BiGS reconstructs view- and light-dependent color functions for Gaussian Splats using OLAT data such as the shown translucent Dragon of 31,252 bidirectional Gaussian primitives. Our model decomposes the appearance of each primitive into different intrinsic components and is able to achieve plausible relighting and novel view synthesis with various environment maps and point light sources.
• Figure 2: Pipeline overview: our method introduces per-Gaussian optimizable lighting parameters: $\mathcal{T}_\mathrm{dir}$, $\mathcal{T}_\mathrm{ind}$, $\rho$, and $s$, each represented using spherical harmonics. Given novel lighting conditions, we relight each Gaussian by generating the view-dependent color of each Gaussian $L(\omega_\mathrm{o})$ represented by spherical harmonics that are compatible with the Gaussian rasterization pipeline, and therefore can render under novel light and view conditions.
• Figure 3: Point light relighting: The leftmost shows the six positions of the point light sources we use to illuminate a translucent Dragon. The renderings of the model illuminated from each position are shown. Dragon becomes brighter as the light gets nearer; please see the supplementary video.
• Figure 4: Example images of our capture OLAT dataset.
• Figure 5: Directional light and environment map relighting. The Knob model is lit with a vertical directional light. The FurBall model is lit using an environment map. Both are viewed from three different angles.