Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering

TL;DR

This work introduces SVRaster, a neural-free radiance-field rendering framework that uses a multi-level sparse voxel grid and a specialized rasterizer to achieve real-time rendering with high fidelity. By employing an octree-based sparse voxel representation and a direction-dependent Morton ordering, the method preserves correct depth ordering and eliminates popping artifacts without resorting to dense 3D grids or neural networks. The approach combines explicit voxel densities and SH-based view-dependent colors, enabling seamless integration with grid-based 3D processing tools such as Volume Fusion and Marching Cubes for mesh extraction. Across novel-view synthesis and mesh reconstruction tasks, SVRaster achieves state-of-the-art-like performance with competitive quality, while offering substantial speedups and flexibility for future extensions and applications. The results demonstrate practical impact for real-time rendering, scalable scene representation, and compatibility with classical 3D processing pipelines in radiance-field applications.

Abstract

We propose an efficient radiance field rendering algorithm that incorporates a rasterization process on adaptive sparse voxels without neural networks or 3D Gaussians. There are two key contributions coupled with the proposed system. The first is to adaptively and explicitly allocate sparse voxels to different levels of detail within scenes, faithfully reproducing scene details with $65536^3$ grid resolution while achieving high rendering frame rates. Second, we customize a rasterizer for efficient adaptive sparse voxels rendering. We render voxels in the correct depth order by using ray direction-dependent Morton ordering, which avoids the well-known popping artifact found in Gaussian splatting. Our method improves the previous neural-free voxel model by over 4db PSNR and more than 10x FPS speedup, achieving state-of-the-art comparable novel-view synthesis results. Additionally, our voxel representation is seamlessly compatible with grid-based 3D processing techniques such as Volume Fusion, Voxel Pooling, and Marching Cubes, enabling a wide range of future extensions and applications.

Figures (13)

• Figure 1: We propose SVRaster, a novel framework for multi-view reconstruction and novel view synthesis. (a) Sparse voxel representation effectively captures the volume density and radiance field of the scene, without the need for neural networks, 3D Gaussians, and sparse-points prior. (b) Using our customized sparse voxel rasterizer, we can learn the underlying 3D scene efficiently and achieve state-of-the-art performance in both rendering quality and speed. (c) Notably, lifting 2D modal to the trained sparse voxels is simple and efficient by integrating the classic Volume Fusion volumefusionkinectfusionbundlefusion. We show examples of vision foundation model feature field from RADIO radio, semantic field from Segformer segformer, and signed distance field from rendered depth, making it flexible and suitable for a wide range of applications.
• Figure 2: Sparse voxels scene representation.(Left) We allocate voxel under an Octree layout. Each voxels has its own Spherical Harmonic coefficient for view-dependent appearance. The color field is approximated as a constant inside a voxel when rendering a view for efficiency. The density field is trilienarly varied inside a voxel and is modeled by the density values on the corner grid points (i.e., the black dots $\bullet$) of each voxel. The grid points densities are shared between adjacent voxels. (Right) We evenly sample $K$ points inside the segment of ray-voxel intersection to compute volume integration for its alpha value contributing to the pixel ray. See \ref{['sssec:scene_repr']} for details.
• Figure 3: Rasterization procedure. Refer to \ref{['sssec:rasterization']} for details.
• Figure 4: Illustration of the rendering order.(a) In both cases, the smaller voxels should be rendered first but they will arranged behind the larger voxels if using voxel centers or the nearest corners as the sorting order. (b) We show the four types of Morton order under the 2D world. The voxel rendering order under an Octree node is depend on which world quadrant the ray direction is pointing to. (c) An toy example of sorting the Morton order encoding. All the ray directions going toward the up-right quadrant can use the sorted voxels for a correct rendering order. See \ref{['sssec:rasterization']} for details.
• Figure 5: Visualization of voxel sampling rate and grid layout initialization.(a) We visualize the voxel sampling rate defined in \ref{['eq:sampling_rate']}. (b) We depict the foreground main region and the background region under different shell levels. In unbounded scenes, we apply different grid layout initialization strategies for foreground and background regions. See \ref{['ssec:scene_init']} for details.