GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering

TL;DR

GI-GS presents a 3D Gaussian Splatting-based inverse rendering framework that decomposes global illumination by combining a G-buffer-driven deferred shading pipeline for direct lighting with path-tracing-based indirect illumination. By augmenting Gaussians with per-point normals and BRDF attributes and employing a tile-based, efficient path tracing approach, the method achieves accurate geometry, materials, and relighting under unknown illumination. It extends 3DGS to world-space indirect lighting using cubemaps to recover global geometry and leverages a learnable environment map for direct lighting, enabling high-fidelity novel view synthesis and competitive relighting performance with improved efficiency. Across TensoIR and Mip-NeRF 360 datasets, GI-GS demonstrates strong quantitative gains and visually faithful lighting interactions, highlighting its practical impact for inverse rendering and scene relighting tasks.

Abstract

We present GI-GS, a novel inverse rendering framework that leverages 3D Gaussian Splatting (3DGS) and deferred shading to achieve photo-realistic novel view synthesis and relighting. In inverse rendering, accurately modeling the shading processes of objects is essential for achieving high-fidelity results. Therefore, it is critical to incorporate global illumination to account for indirect lighting that reaches an object after multiple bounces across the scene. Previous 3DGS-based methods have attempted to model indirect lighting by characterizing indirect illumination as learnable lighting volumes or additional attributes of each Gaussian, while using baked occlusion to represent shadow effects. These methods, however, fail to accurately model the complex physical interactions between light and objects, making it impossible to construct realistic indirect illumination during relighting. To address this limitation, we propose to calculate indirect lighting using efficient path tracing with deferred shading. In our framework, we first render a G-buffer to capture the detailed geometry and material properties of the scene. Then, we perform physically-based rendering (PBR) only for direct lighting. With the G-buffer and previous rendering results, the indirect lighting can be calculated through a lightweight path tracing. Our method effectively models indirect lighting under any given lighting conditions, thereby achieving better novel view synthesis and competitive relighting. Quantitative and qualitative results show that our GI-GS outperforms existing baselines in both rendering quality and efficiency.

Figures (31)

• Figure 1: Overview of GI-GS. GI-GS takes input a set of pretrianed 3D Gaussians, each with a normal attribute. It first rasterizes the scene geometry and materials into a G-buffer. Next, it incorporates a differentiable PBR pipeline to obtain the rendering result under direct lighting and performs path tracing to model the occlusion. Finally, it employs differentiable ray tracing to calculate indirect lighting from the scene geometry and the previous rendering result. The final rendered image is a fusion of the first-pass and second-pass results and uses the ground truth image for supervision.
• Figure 2: (a) Occlusion quantifies the degree to which a surface point can receive ambient light. Points on flat surfaces demonstrate high visibility due to less occlusion from surrounding geometry. In contrast, points located in holes, corners, or adjacent to other surfaces appear darker, since they are more occluded. (b) Path tracing: For each surface point corresponding to a pixel on the depth map, we perform ray marching starting from that point to calculate the visibility in different directions.
• Figure 3: (a) When the surface point ${\bm{x}}$ is occluded by $\boldsymbol{\hat{x}}$ in direction $\boldsymbol{\omega}_i$, $\boldsymbol{\hat{x}}$ acts as an indirect light source. In this case, the light received by ${\bm{x}}$ in direction $\boldsymbol{\omega}_i$ is equivalent to the light from the actual light source reflected by $\boldsymbol{\hat{x}}$ in the $-\boldsymbol{\omega}_i$ direction. (b) To obtain global geometry and lighting information, we rotate the camera to render the results of the remaining five perspectives and form a cubemap. This cubemap is then utilized to restore the global geometry and facilitate path tracing.
• Figure 4: Qualitative comparison on the TensoIR dataset. We visualize the rendering results, reconstructed albedo, and relighting results, respectively.
• Figure 5: Qualitative comparison on the Mip-NeRF 360 dataset. Our GI-GS effectively reconstructs high-frequency details and occluded areas in real-world scenes. Notably, the extended version, Ours-Cubemap, achieves a smoother result in occluded areas.