Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis

Alexander Mai, Peter Hedman, George Kopanas, Dor Verbin, David Futschik, Qiangeng Xu, Falko Kuester, Jonathan T. Barron, Yinda Zhang

TL;DR

EVER addresses real-time view synthesis by introducing exact blending of overlapping ellipsoidal primitives within a ray-traced, differentiable volume renderer. It replaces opacity-based splats with constant-density ellipsoids and computes the volume rendering integral exactly via double intersections in a ray-tracing pipeline, with view-dependent color encoded by spherical harmonics. The method leverages adaptive density control and per-ray BVH sorting to render at about 30 FPS at 720p on an RTX 4090, achieving state-of-the-art sharpness on Zip-NeRF among real-time methods and competitive results on Mip-NeRF 360, while maintaining 3D-consistency on large scenes. By demonstrating improved blending and sharpness through density-based primitives and exact interval-wise accumulation, EVER narrows the gap between offline radiance-field methods and real-time rendering while enabling optical effects like defocus and fisheye distortion.

Abstract

We present Exact Volumetric Ellipsoid Rendering (EVER), a method for real-time differentiable emission-only volume rendering. Unlike recent rasterization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation allows for exact volume rendering, rather than alpha compositing 3D Gaussian billboards. As such, unlike 3DGS our formulation does not suffer from popping artifacts and view dependent density, but still achieves frame rates of $\sim\!30$ FPS at 720p on an NVIDIA RTX4090. Since our approach is built upon ray tracing it enables effects such as defocus blur and camera distortion (e.g. such as from fisheye cameras), which are difficult to achieve by rasterization. We show that our method is more accurate with fewer blending issues than 3DGS and follow-up work on view-consistent rendering, especially on the challenging large-scale scenes from the Zip-NeRF dataset where it achieves sharpest results among real-time techniques.

EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis

TL;DR

EVER addresses real-time view synthesis by introducing exact blending of overlapping ellipsoidal primitives within a ray-traced, differentiable volume renderer. It replaces opacity-based splats with constant-density ellipsoids and computes the volume rendering integral exactly via double intersections in a ray-tracing pipeline, with view-dependent color encoded by spherical harmonics. The method leverages adaptive density control and per-ray BVH sorting to render at about 30 FPS at 720p on an RTX 4090, achieving state-of-the-art sharpness on Zip-NeRF among real-time methods and competitive results on Mip-NeRF 360, while maintaining 3D-consistency on large scenes. By demonstrating improved blending and sharpness through density-based primitives and exact interval-wise accumulation, EVER narrows the gap between offline radiance-field methods and real-time rendering while enabling optical effects like defocus and fisheye distortion.

Abstract

We present Exact Volumetric Ellipsoid Rendering (EVER), a method for real-time differentiable emission-only volume rendering. Unlike recent rasterization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation allows for exact volume rendering, rather than alpha compositing 3D Gaussian billboards. As such, unlike 3DGS our formulation does not suffer from popping artifacts and view dependent density, but still achieves frame rates of FPS at 720p on an NVIDIA RTX4090. Since our approach is built upon ray tracing it enables effects such as defocus blur and camera distortion (e.g. such as from fisheye cameras), which are difficult to achieve by rasterization. We show that our method is more accurate with fewer blending issues than 3DGS and follow-up work on view-consistent rendering, especially on the challenging large-scale scenes from the Zip-NeRF dataset where it achieves sharpest results among real-time techniques.
Paper Structure (3 sections, 1 equation, 2 figures)

This paper contains 3 sections, 1 equation, 2 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Motivation

Figures (2)

  • Figure 1: An overview of the quality benefits of our EVER technique. Left: On the Zip-NeRF dataset, our model produces the sharpest results ever. Middle: Our method is able to blend primitive colors together to reproduce light and shadow better than other 3DGS-based methods. Right: The way our approach blends colors is physically accurate over any number of primitives, matching the ground truth computed via quadrature.
  • Figure 2: (a) Here we show a simple "flatland" scene containing two primitives (one red, one blue) with a camera orbiting them, viewed from above. We render this orbit using three different techniques, where each camera position yields a one-dimensional "image" (a scanline) which are stacked vertically to produce these epipolar plane image (EPI) visualizations. (b, c) The approximations made by approximate splatting-based techniques result in improper blending due to discontinuities, which are visible as horizontal lines across the EPI. In contrast, (d) our method's exact rendering yields a smooth EPI, with bands of purple from color blending.