Radiant Foam: Real-Time Differentiable Ray Tracing

TL;DR

Radiant Foam introduces a differentiable, Voronoi-based volumetric mesh (a foam) for real-time ray-traced rendering without rasterization. By parameterizing space with differentiable Voronoi sites and leveraging a ray-tracing traversal akin to ray_plane, it achieves NeRF-like optimization while preserving explicit geometry and true light transport. The approach matches the quality of Gaussian Splatting on challenging scenes but delivers substantially higher frame rates and hardware‑agnostic performance. Extensive experiments on Mip-NeRF360 and Deep Blending show competitive image quality with up to ~300 FPS, and ablations highlight the importance of densification, pruning, and the quantile loss. Limitations include the current Voronoi constraint on cell boundaries, with future work aimed at generalizing topology and handling dynamic content.

Abstract

Research on differentiable scene representations is consistently moving towards more efficient, real-time models. Recently, this has led to the popularization of splatting methods, which eschew the traditional ray-based rendering of radiance fields in favor of rasterization. This has yielded a significant improvement in rendering speeds due to the efficiency of rasterization algorithms and hardware, but has come at a cost: the approximations that make rasterization efficient also make implementation of light transport phenomena like reflection and refraction much more difficult. We propose a novel scene representation which avoids these approximations, but keeps the efficiency and reconstruction quality of splatting by leveraging a decades-old efficient volumetric mesh ray tracing algorithm which has been largely overlooked in recent computer vision research. The resulting model, which we name Radiant Foam, achieves rendering speed and quality comparable to Gaussian Splatting, without the constraints of rasterization. Unlike ray traced Gaussian models that use hardware ray tracing acceleration, our method requires no special hardware or APIs beyond the standard features of a programmable GPU.

Figures (8)

• Figure 1: Teaser -- We introduce the Radiant Foam differentiable 3D representation, which can be used to learn accurate radiance fields for any novel view synthesis applications (left). If we slice our foam along the plane highlighted by the red "laser", we expose (right) the internal structure of our representation: a polyhedral mesh that provides an injective parameterization of the 3D domain. Our representation is a foam, as the polyhedral cell structure is analogous to a closed-cell foam which partitions space into regions physically separated by thin, flat walls. It is radiant, as each foam bubble emits a view-dependent radiance that can be used to model the plenoptic function.
• Figure 2: Radiant Foam -- (left) In a stable foam, the pressure inside each bubble is roughly equal. The interfaces between bubbles settle into shapes that balance forces, leading to polygonal cells resembling Voronoi patterns (right). Our representation is nothing but a dense Voronoi tessellation of 3D space, where each point belongs to exactly one Voronoi cell. The position of Voronoi sites is differentiable, making it amenable to gradient-based optimization.
• Figure 3: Ray-based effects -- Ray tracing simplifies the implementation of many effects which are difficult to approximate with rasterization. To motivate our work, we show here examples of integrating reflections, refractions, and non-linear camera models into our rendering pipeline. Each would be complicated to achieve with rasterization, but requires only minor modification to our rendering code.
• Figure 4: Ray tracing foams -- When a ray (red) enters a cell, we iterate through all the planar cell faces to identify the face through which the ray exits. This exit intersection is the first intersection along the ray with a normal vector less than 90 degrees from the ray direction (green); other intersections are considered back-facing (blue) and ignored. As the faces each correspond to a neighboring cell, the tracing then proceeds by stepping into the cell associated with the exit intersection and repeating the process.
• Figure 5: Delaunay and its dual Voronoi -- (left) Given a set of points (red) in $\mathbb{R}^N$, we can find circumspheres (blue) which each pass through $N{+}1$ points. (center) The set of all circumspheres which contain no points on their interior defines the Delaunay triangulation, where the $N{+}1$ points tangent to each circumsphere form a simplex. (right) These circumspheres also describe the Delaunay triangulation's dual, the Voronoi diagram: the centers of circumspheres tangent to a point become the vertices of the Voronoi cell containing that point.