Residual path integrals for re-rendering

TL;DR

The paper introduces the residual path integral as an unbiased, efficient framework for incremental re-rendering when scenes contain moving objects or material edits. By decomposing the light-transport problem into a residual between frames and isolating a dynamic path space, the authors design specialized sampling strategies and path mappings that focus computation on the portion of path space affected by changes. They demonstrate substantial speedups over full re-rendering and correlated path tracing, with practical benefits in editing workflows and scene animation. The work also establishes a foundation for integrating advanced MIS-based sampling and gradient-like path mappings, enabling further exploration of sampling trade-offs and extensions to deformable scenes and participating media.

Abstract

Conventional rendering techniques are primarily designed and optimized for single-frame rendering. In practical applications, such as scene editing and animation rendering, users frequently encounter scenes where only a small portion is modified between consecutive frames. In this paper, we develop a novel approach to incremental re-rendering of scenes with dynamic objects, where only a small part of a scene moves from one frame to the next. We formulate the difference (or residual) in the image between two frames as a (correlated) light-transport integral which we call the residual path integral. Efficient numerical solution of this integral then involves (1)~devising importance sampling strategies to focus on paths with non-zero residual-transport contributions and (2)~choosing appropriate mappings between the native path spaces of the two frames. We introduce a set of path importance sampling strategies that trace from the moving object(s) which are the sources of residual energy. We explore path mapping strategies that generalize those from gradient-domain path tracing to our importance sampling techniques specially for dynamic scenes. Additionally, our formulation can be applied to material editing as a simpler special case. We demonstrate speed-ups over previous correlated sampling of path differences and over rendering the new frame independently. Our formulation brings new insights into the re-rendering problem and paves the way for devising new types of sampling techniques and path mappings with different trade-offs.

Figures (12)

• Figure 1: Examples of dynamic paths. We show three-vertex paths involving a moving triangle. In each scene, the location of the triangle in the other scene is in light green. In the new scene, we can see two types of "dynamic" paths. One hits the dynamic triangle (path in red) while the other one passes through the ghost triangle (in blue; as per the position of the dynamic counterpart in the old scene). The red and blue paths did not exist in the previous scene (thus shown as dashed lines), so they would contribute to the difference integral. The black paths represent static paths that do not contribute to the difference path integral.
• Figure 2: Dynamic-path sampling techniques. Row A illustrates six techniques plus the regular path tracing with non-affected paths rejected (\ref{['sec:dynamic_path_sampling']}). We move the small box slightly to the right in the Cornell Box scene (a, b). We render the difference radiance contribution from each sampling technique separately in Row B. Row C is Row B weighted by the corresponding multiple importance sampling weight for each column. A sum of Row C gives the total contribution by dynamic paths in (c). (d) An example for multiple importance sampling computation in \ref{['subsec:pdf_computation']}, where a dynamic path hits dynamic objects once and ghost objects 4 times.
• Figure 3: (a) Illustration of pdf construction for our set of dynamic path sampling strategies. Indices match \ref{['fig:importance_sample']}. (b) An MIS example in terms of pdfs showing the same path can be sampled by two techniques. (c) Geometries used for change of measure when computing pdfs for dynamic paths starting from a ghost vertex. We represent dynamic edges passing through ghost objects by dashed segments.
• Figure 4: Path mappings for various dynamic sampling techniques. On the left of each pair, we sample the base path (black) using our methods shown in \ref{['fig:importance_sample']}. Mapped paths are blue on the right. Two frames can be flipped for the two-way path mapping.
• Figure 5: Equal-time comparison. We edit the material of the book (marked in yellow box) below a lamp shedding white light. The albedo is edited from blue to white, thereby the book is reflecting differently shaded light into the scene and largely changing the global illumination. Path tracing: 64spp. Correlated PT: 32spp. Incremental(Ours): 10spp*3; Reference for each frame: 6400spp. Please zoom in for better inspection of noise.