LinPrim: Linear Primitives for Differentiable Volumetric Rendering

TL;DR

LinPrim presents a differentiable volumetric rendering framework that uses simple, bounded linear primitives—octahedra and tetrahedra—to reconstruct 3D scenes with real-time rendering. The method combines a GPU-friendly differentiable rasterizer, a preprocessing-and-rasterization pipeline, anti-aliasing, and gradient-based optimization (L1+SSIM) with population control and LinPrim-MCMC for dynamic primitive management. Experiments on real-world datasets (e.g., ScanNet++ and Mip-NeRF 360) show that LinPrim achieves competitive reconstruction fidelity using far fewer primitives than Gaussian-based methods, with octahedra generally offering stronger stability. This work expands the design space of 3D representations for novel view synthesis and paves the way for hybrid or mesh-bridging approaches in scalable, explicit scene representations.

Abstract

Volumetric rendering has become central to modern novel view synthesis methods, which use differentiable rendering to optimize 3D scene representations directly from observed views. While many recent works build on NeRF or 3D Gaussians, we explore an alternative volumetric scene representation. More specifically, we introduce two new scene representations based on linear primitives - octahedra and tetrahedra - both of which define homogeneous volumes bounded by triangular faces. To optimize these primitives, we present a differentiable rasterizer that runs efficiently on GPUs, allowing end-to-end gradient-based optimization while maintaining real-time rendering capabilities. Through experiments on real-world datasets, we demonstrate comparable performance to state-of-the-art volumetric methods while requiring fewer primitives to achieve similar reconstruction fidelity. Our findings deepen the understanding of 3D representations by providing insights into the fidelity and performance characteristics of transparent polyhedra and suggest that adopting novel primitives can expand the available design space.

Figures (6)

• Figure 1: We present LinPrim, a new take on novel view synthesis, that leverages linear primitives - octahedra and tetrahedra - for differentiable volumetric rendering in order to facilitate 3D scene reconstruction. To this end, we propose a differentiable rendering pipeline in conjunction with a real-time capable rasterizer tailored to such primitives, thus achieving interactive frame rates for novel-view rendering and showcasing the potential of polyhedral primitives in NVS workflows.
• Figure 2: Overview of our method. Primitives are constructed from SfM points. During rendering, primitives are preprocessed according to the camera pose and sorted front-to-back. Afterward, the list is traversed and the individual color contributions are blended for each pixel. Through comparison with a known view, the primitives' features and the population are adjusted based on gradient flow.
• Figure 3: Visualization of our scene representation. We show all octahedra with an opacity of at least $0.25$ on an optimized Mip-NeRF 360 barron_mip-nerf_2022 Bicycle scene reconstruction.
• Figure 4: Qualitative Results on test views from ScanNet++ v2 scenes yeshwanth_scannet_2023.
• Figure 5: Qualitative depth comparison on ScanNet++ v2 scenes. Predicted depth maps from LinPrim and 3DGS are shown alongside depths rendered from ground truth meshes.