Unified Gaussian Primitives for Scene Representation and Rendering

TL;DR

This work formulate the rendering theory for the primitive based on non-exponential transport and derive efficient rendering operations to be compatible with Monte Carlo path tracing and demonstrate the versatility of the representation in various rendering applications such as global illumination and appearance editing, while supporting arbitrary lighting conditions by nature.

Abstract

Searching for a unified scene representation remains a research challenge in computer graphics. Traditional mesh-based representations are unsuitable for dense, fuzzy elements, and introduce additional complexity for filtering and differentiable rendering. Conversely, voxel-based representations struggle to model hard surfaces and suffer from intensive memory requirement. We propose a general-purpose rendering primitive based on 3D Gaussian distribution for unified scene representation, featuring versatile appearance ranging from glossy surfaces to fuzzy elements, as well as physically based scattering to enable accurate global illumination. We formulate the rendering theory for the primitive based on non-exponential transport and derive efficient rendering operations to be compatible with Monte Carlo path tracing. The new representation can be converted from different sources, including meshes and 3D Gaussian splatting, and further refined via transmittance optimization thanks to its differentiability. We demonstrate the versatility of our representation in various rendering applications such as global illumination and appearance editing, while supporting arbitrary lighting conditions by nature. Additionally, we compare our representation to existing volumetric representations, highlighting its efficiency to reproduce details.

Figures (14)

• Figure 1: In conventional volume path tracing, a collision point is sampled at each time and the apparent appearance of a primitive depends on multiple scattering (left). Our method samples the entire primitive and uses the per-primitive phase function to scatters light once, essentially treating it as an indivisible entity.
• Figure 2: 2D flatland illustration of a ray traveling through a heterogeneous volume consisting of 5 Gaussian primitives (top) and the corresponding free-flight CDF and PDF (bottom). The CDF is accumulated from $0$ to $1$, where it reaches the saturating distance $t_{\mathrm{sat}}$. The ray is effectively "blocked" at this point, and the PDF is 0 afterwards.
• Figure 3: Illustration for free-flight distribution sampling. (a) If the primitives do not touch $t = t_{\mathrm{sat}}$, it is safe to invert the free-flight CDF by accumulating the per-primitive contributions and check the interval $u$ falls within. Overlapped primitives are automatically handled. (b) However, when primitives are ambiguous, doing so will skew the free-flight distribution. Here, $G_4$ will never be sampled. (c) Instead, we need to perform a disambiguation step by first solving for $t_u$ and then re-sample the primitives based on the clamped distributions.
• Figure 4: Even when there are multiple primitives in a leaf node, it is possible that not all of them are ambiguous. We use line sweeping to find the exact ambiguous segment (highlighted by the red box).
• Figure 5: Rendering 1K random Gaussians assigned with random colors using our free-flight sampling algorithm with different samples per pixel (spp). Difference images against the reference are provided.