GlossGau: Efficient Inverse Rendering for Glossy Surface with Anisotropic Spherical Gaussian

TL;DR

GlossGau addresses the challenge of efficiently reconstructing glossy scenes by extending 3D Gaussian Splatting with an explicit, per-Gaussian anisotropic BRDF model using Anisotropic Spherical Gaussian (ASG). It employs surfel-based Gaussian primitives and a three-stage training strategy, coupled with regularizations for normals and lighting, to decouple geometry and material properties without sacrificing speed. The approach achieves competitive or superior fidelity on diverse datasets while significantly reducing optimization time and storage compared with prior GS-based inverse rendering methods, enabling near real-time rendering. This work advances practical inverse rendering for glossy materials by combining efficient geometry representation, accurate lighting estimation, and robust normal estimation.

Abstract

The reconstruction of 3D objects from calibrated photographs represents a fundamental yet intricate challenge in the domains of computer graphics and vision. Although neural reconstruction approaches based on Neural Radiance Fields (NeRF) have shown remarkable capabilities, their processing costs remain substantial. Recently, the advent of 3D Gaussian Splatting (3D-GS) largely improves the training efficiency and facilitates to generate realistic rendering in real-time. However, due to the limited ability of Spherical Harmonics (SH) to represent high-frequency information, 3D-GS falls short in reconstructing glossy objects. Researchers have turned to enhance the specular expressiveness of 3D-GS through inverse rendering. Yet these methods often struggle to maintain the training and rendering efficiency, undermining the benefits of Gaussian Splatting techniques. In this paper, we introduce GlossGau, an efficient inverse rendering framework that reconstructs scenes with glossy surfaces while maintaining training and rendering speeds comparable to vanilla 3D-GS. Specifically, we explicitly model the surface normals, Bidirectional Reflectance Distribution Function (BRDF) parameters, as well as incident lights and use Anisotropic Spherical Gaussian (ASG) to approximate the per-Gaussian Normal Distribution Function under the microfacet model. We utilize 2D Gaussian Splatting (2D-GS) as foundational primitives and apply regularization to significantly alleviate the normal estimation challenge encountered in related works. Experiments demonstrate that GlossGau achieves competitive or superior reconstruction on datasets with glossy surfaces. Compared with previous GS-based works that address the specular surface, our optimization time is considerably less.

Figures (7)

• Figure 1: GlossGau preserves the real-time rendering capabilities of 3D-GS while producing photorealistic images across diverse surface materials. Our method successfully captures glossy highlights and shadowing without compromising the quality of surrounding areas. GlossGau utilizes less storage space and optimizes at faster speed compared with jiang2024gaussianshader and gao2023relightable.
• Figure 2: GlossGau Pipeline. Our framework initializes with surfel-based Gaussian primitives, where each primitive carries both geometric attributes (position, covariance, and opacity) and appearance properties. The appearance representation decomposes into view-dependent diffuse albedo, parameterized via Spherical Harmonics, and specular BRDF terms, modeled through Anisotropic Spherical Gaussians. The final rendering integrates these material maps with differentiable environment lighting to achieve high-fidelity inverse rendering results.
• Figure 3: Visualized normal results on Shiny Blender dataset verbin2022ref. It illustrates the superior normal estimation achieved by our method.
• Figure 4: The qualitative comparisons on NeRF Synthetic Dataset mildenhall2020nerf. Our method renders the glossy surfaces with high fidelity and reconstructs accurate shadowing. Some areas are zoomed in for better visualization.
• Figure 5: Visualized normals on NeRF Synthetic Dataset mildenhall2020nerf.