Reflective Gaussian Splatting

TL;DR

This work tackles the challenge of real-time, high-quality rendering and reconstruction of reflective scenes with inter-reflection. It introduces Reflective Gaussian Splatting (Ref-Gaussian), which combines physically based deferred rendering with pixel-level BRDFs and a Gaussian-grounded inter-reflection model within a 2D Gaussian splatting framework, augmented by per-Gaussian shading, material-aware normal propagation, and TSDF-based visibility. A key technical element is the split-sum approximation, $L_s(ω_o) ≈ (∫Ω f_s(ω_i, ω_o)(ω_i·N) dω_i)(∫Ω L_i(ω_i) D(ω_i, ω_o)(ω_i·N) dω_i)$, enabling efficient, accurate handling of specular lighting and inter-reflections; ray-traced visibility on a mesh handles occlusion for indirect lighting. The approach achieves state-of-the-art results on reflective datasets, accelerates training and rendering, and supports relighting and editing, offering a unified solution that also generalizes to non-reflective scenes. Overall, Ref-Gaussian provides a practical, geometry-aware, BRDF-consistent framework for rapid, high-fidelity scene reconstruction and rendering with inter-reflection effects.

Abstract

Novel view synthesis has experienced significant advancements owing to increasingly capable NeRF- and 3DGS-based methods. However, reflective object reconstruction remains challenging, lacking a proper solution to achieve real-time, high-quality rendering while accommodating inter-reflection. To fill this gap, we introduce a Reflective Gaussian splatting (Ref-Gaussian) framework characterized with two components: (I) Physically based deferred rendering that empowers the rendering equation with pixel-level material properties via formulating split-sum approximation; (II) Gaussian-grounded inter-reflection that realizes the desired inter-reflection function within a Gaussian splatting paradigm for the first time. To enhance geometry modeling, we further introduce material-aware normal propagation and an initial per-Gaussian shading stage, along with 2D Gaussian primitives. Extensive experiments on standard datasets demonstrate that Ref-Gaussian surpasses existing approaches in terms of quantitative metrics, visual quality, and compute efficiency. Further, we show that our method serves as a unified solution for both reflective and non-reflective scenes, going beyond the previous alternatives focusing on only reflective scenes. Also, we illustrate that Ref-Gaussian supports more applications such as relighting and editing.

Figures (14)

• Figure 1: Our Ref-Gaussian achieves superior rendering quality in novel view synthesis while enjoying fast optimization (training time) and real-time rendering (FPS), and supporting various downstream applications such as relighting and editing. Please refer to the video in our supplementary material for a more comprehensive and intuitive comparison.
• Figure 2: Overview of the Ref-Gaussian framework: First, we apply the splatting process to produce feature maps and perform ray-tracing on the extracted mesh to compute visibility for the specular term in the rendering equation. Next, we use the pixel-level feature maps to apply the rendering equation with split-sum approximation, yielding the final physically based rendering result.
• Figure 3: Qualitative comparisons on reflective scenes, including Bell, Luyu and Helmet.
• Figure 4: Qualitative comparisons on Ref-Real dataset verbin2022ref.
• Figure 5: Inverse rendering with extracted mesh and indirect light. Indirect: Only consider indirect light as specular component when rendering.