Hybrid Rendering for Dynamic Scenes

TL;DR

This work tackles real-time global illumination for scenes with static environments and dynamic elements by introducing a hybrid rendering pipeline that precomputes or learns the static light transport $L_s$ and then renders a sparse dynamic delta $L_\\Delta = L_+ - L_-$ via adaptive sampling and a primary-sample-space formulation. The static component is represented by a hash-grid + MLP regression trained offline to produce $\\tilde{L}_s$, while dynamic contributions are computed through additive and subtractive passes, yielding the final image $L_H = L_s + L_\\Delta$. Key contributions include a mathematical justification of unbiasedness under the delta framework, the use of primary sample space to preserve noise correlation, adaptive sampling focused on dynamic regions, and a masking strategy to mitigate learned-static biases, with support for dynamic lights and environment maps. The approach achieves real-time performance (30–50 fps) in scenes with static levels and moving content, reduces noise significantly, and integrates well with denoisers and path-guiding techniques, offering a practical path to high-quality GI in games.

Abstract

Despite significant advances in algorithms and hardware, global illumination continues to be a challenge in the real-time domain. Time constraints often force developers to either compromise on the quality of global illumination or disregard it altogether. We take advantage of a common setup in modern games: having a set of a level, which is a static scene with dynamic characters and lighting. We introduce a novel method for efficiently and accurately rendering global illumination in dynamic scenes. Our hybrid technique leverages precomputation and neural networks to capture the light transport of a static scene. Then, we introduce a method to compute the difference between the current scene and the static scene, which we already precomputed. By handling the bulk of the light transport through precomputation, our method only requires the rendering of a minimal difference, reducing the noise and increasing the quality.

Figures (6)

• Figure 1: Example of different paths during rendering a scene. Here, we have a beige sphere as a dynamic object. The blue path $\in \mathbf{\Omega}_o$ doesn't interact with the dynamic object. The green path $\in \mathbf{\Omega}_+$ reflects from the dynamic object. And the red path $\in \mathbf{\Omega}_-$ goes directly through the dynamic object unaffected.
• Figure 2: Example of hybrid rendering with and without the mask. Without the mask, we get visible artifacts at the edges due to the difference between the real base image and the learned-based image. The background error is leaking through the dynamic objects. Due to the nature of artifacts, it's hard for a denoiser to get rid of them.
• Figure 3: Example of our method with a denoiser applied. The 1st column is path-traced images with 2 spp. The 2nd column is our method with just one 1spp. The next to last column is our method with adaptive sampling. Even without the adaptive sampling, our method is much better compared to the path-tracer. Our method lowers the noise level in the input to the denoiser, which improves the quality of the denoiser.
• Figure 4: Scenes illuminated by changing environment lighting. In path tracing, we need to render everything from scratch. In our method, we only render the difference, leading to less noise.
• Figure 5: Example of our method on different scenes. For each scene, we added a new dynamic object. The original scenes are labeled as static. The 1st column shows path-traced results with 2spp. The next column shows our results with 1spp. It already has 3 to 6 times better quality. The second to last column is our method with adaptive sampling. By redistributing samples to the region of change, we achieve up to 8 times improvement.