Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NieR: Normal-Based Lighting Scene Rendering

Hongsheng Wang, Yang Wang, Yalan Liu, Fayuan Hu, Shengyu Zhang, Fei Wu, Feng Lin

TL;DR

NieR addresses dynamic lighting and material diversity in road-scene rendering by extending 3D Gaussian Splatting with two key modules: Light Decomposition (LD), which splits outgoing radiance into diffuse and specular components using surface normals and a specular coefficient $a$, and Hierarchical Normal Gradient Densification (HNGD), which adaptively densifies Gaussian points based on gradient information via Grad = $(1-\omega)\cdot (G_{xyz}/denom) + \omega\cdot (G_{norm}/denom)$. The approach leverages a physically inspired rendering framework with equations such as $L_o(\omega_o,x)=\int_{\Omega} f(\omega_o,\omega_i,x)L_i(\omega_i,x)(\omega_i\cdot n) d\omega_i$, and color synthesis $c_0=(1-a)\cdot sh_c \cdot \cos\theta + a\cdot sh_c$, to better capture specular highlights and color variation under dynamic lighting. Extensive experiments on seven Mip-NeRF360 scenes and Tanks & Temples show NieR achieving higher PSNR/SSIM and lower LPIPS than baselines like Gaussian Splatting, with substantially faster training times than some NeRF-based methods, confirming improved visual quality and efficiency. The work provides a practical path for lighting-aware 3D Gaussian rendering in dynamic scenes and offers publicly available code for replication and extension.

Abstract

In real-world road scenes, diverse material properties lead to complex light reflection phenomena, making accurate color reproduction crucial for enhancing the realism and safety of simulated driving environments. However, existing methods often struggle to capture the full spectrum of lighting effects, particularly in dynamic scenarios where viewpoint changes induce significant material color variations. To address this challenge, we introduce NieR (Normal-Based Lighting Scene Rendering), a novel framework that takes into account the nuances of light reflection on diverse material surfaces, leading to more precise rendering. To simulate the lighting synthesis process, we present the LD (Light Decomposition) module, which captures the lighting reflection characteristics on surfaces. Furthermore, to address dynamic lighting scenes, we propose the HNGD (Hierarchical Normal Gradient Densification) module to overcome the limitations of sparse Gaussian representation. Specifically, we dynamically adjust the Gaussian density based on normal gradients. Experimental evaluations demonstrate that our method outperforms state-of-the-art (SOTA) methods in terms of visual quality and exhibits significant advantages in performance indicators. Codes are available at https://wanghongsheng01.github.io/NieR/.

NieR: Normal-Based Lighting Scene Rendering

TL;DR

NieR addresses dynamic lighting and material diversity in road-scene rendering by extending 3D Gaussian Splatting with two key modules: Light Decomposition (LD), which splits outgoing radiance into diffuse and specular components using surface normals and a specular coefficient , and Hierarchical Normal Gradient Densification (HNGD), which adaptively densifies Gaussian points based on gradient information via Grad = . The approach leverages a physically inspired rendering framework with equations such as , and color synthesis , to better capture specular highlights and color variation under dynamic lighting. Extensive experiments on seven Mip-NeRF360 scenes and Tanks & Temples show NieR achieving higher PSNR/SSIM and lower LPIPS than baselines like Gaussian Splatting, with substantially faster training times than some NeRF-based methods, confirming improved visual quality and efficiency. The work provides a practical path for lighting-aware 3D Gaussian rendering in dynamic scenes and offers publicly available code for replication and extension.

Abstract

In real-world road scenes, diverse material properties lead to complex light reflection phenomena, making accurate color reproduction crucial for enhancing the realism and safety of simulated driving environments. However, existing methods often struggle to capture the full spectrum of lighting effects, particularly in dynamic scenarios where viewpoint changes induce significant material color variations. To address this challenge, we introduce NieR (Normal-Based Lighting Scene Rendering), a novel framework that takes into account the nuances of light reflection on diverse material surfaces, leading to more precise rendering. To simulate the lighting synthesis process, we present the LD (Light Decomposition) module, which captures the lighting reflection characteristics on surfaces. Furthermore, to address dynamic lighting scenes, we propose the HNGD (Hierarchical Normal Gradient Densification) module to overcome the limitations of sparse Gaussian representation. Specifically, we dynamically adjust the Gaussian density based on normal gradients. Experimental evaluations demonstrate that our method outperforms state-of-the-art (SOTA) methods in terms of visual quality and exhibits significant advantages in performance indicators. Codes are available at https://wanghongsheng01.github.io/NieR/.
Paper Structure (20 sections, 9 equations, 4 figures, 5 tables)

This paper contains 20 sections, 9 equations, 4 figures, 5 tables.

Table of Contents

  1. Related Work
  2. Traditional Physical Rendering Methods
  3. implicit Rendering Methods
  4. 3D Gaussian-based Methods
  5. Methodology
  6. Preliminary
  7. LD: Light Decomposition 3D Gaussian
  8. HNGD : Hierarchical Normal Gradient Densification
  9. Experimentation
  10. Training Detail
  11. Results and Evaluation Results
  12. Ablation Studies
  13. Light Decomposition
  14. Comparisons with Baseline
  15. Quantitative Results
  16. ...and 5 more sections

Figures (4)

  • Figure 1: The pipeline of our method, with the LD module on the left and the HNGD module on the right
  • Figure 2: Shows the comparison between our method and previous methods, as well as the corresponding ground truth images extracted from the preserved test views. The scenes from top to bottom are the stump, bonsaimcounter, room, garden, kitchen, and cycle from the Mip NeRF360 dataset; And trucks and trains from Tanks and Temples. Non obvious quality differences are marked with arrows/embedded markings.
  • Figure 3: The top left image (a) shows the result of removing diffuse light intensity, while the top right image (b) shows the complete result. The left image (c) shows the result of removing specular reflection, while the right image (d) shows the complete result.
  • Figure 4: From left to right (a) shows Ground Truth, (b) shows no hierarchical densification result, (c) shows sparse splitting hierarchical densification result, and (d) shows dense splitting hierarchical densification result.