Volumetric Rendering with Baked Quadrature Fields

TL;DR

This work introduces Volumetric Rendering with Baked Quadrature Fields, a NeRF-based framework that renders non-opaque scenes in real time by baking quadrature points onto a textured polygon mesh. A learned quadrature field defines zero-crossings that yield mesh intersections used as volumetric samplers, with a deformation-based fine-tuning stage and a final baking step that stores neural features in a texture map for fast ray-tracing on standard GPUs. The approach supports transparent media and volumetric effects while maintaining competitive NeRF-quality rendering at interactive frame rates, demonstrated across synthetic and real datasets with thorough ablations. The method leverages marching cubes on the quadrature field to produce a compact, graphics-friendly representation, enabling easy integration with existing pipelines and hardware-accelerated rendering. Limitations include reliance on NeRF-like representations and occasional misses of very thin structures, suggesting future work on scalability and robustness for ultra-thin geometries.

Abstract

We propose a novel Neural Radiance Field (NeRF) representation for non-opaque scenes that enables fast inference by utilizing textured polygons. Despite the high-quality novel view rendering that NeRF provides, a critical limitation is that it relies on volume rendering that can be computationally expensive and does not utilize the advancements in modern graphics hardware. Many existing methods fall short when it comes to modelling volumetric effects as they rely purely on surface rendering. We thus propose to model the scene with polygons, which can then be used to obtain the quadrature points required to model volumetric effects, and also their opacity and colour from the texture. To obtain such polygonal mesh, we train a specialized field whose zero-crossings would correspond to the quadrature points when volume rendering, and perform marching cubes on this field. We then perform ray-tracing and utilize the ray-tracing shader to obtain the final colour image. Our method allows an easy integration with existing graphics frameworks allowing rendering speed of over 100 frames-per-second for a $1920\times1080$ image, while still being able to represent non-opaque objects.

Figures (11)

• Figure 1: We propose using textured polygons with NeRF to efficiently render non-opaque scenes, combining high-quality rendering with modern graphics hardware. To model a scene, we produce a mesh that gives quadrature points along a ray (shown as points on the intersection with the cross-section of the mesh) required in volumetric rendering.
• Figure 2: Overview of our pipeline. We start with a pre-trained network to train a quadrature field that learns the placement of quadrature points. The extracted mesh from the quadrature field is fine-tuned using a deformation field (deformation is shown using red colour on the deformed mesh). Lastly, the neural features are baked into a texture map and the mesh, which can be rendered using ray-tracing.
• Figure 3: Effect of omega on quadrature field. a) quadrature field along a cross-section of a shape, b) zero-crossings of the quadrature field at $\omega$ = 1 and c) at $\omega$ = 50. The higher values of omega leads to more zero-crossings.
• Figure 4: Quadrature field loss. For a particular point, the quad field is supervised to predict the directional gradient to be equal to the maximum weight between the bi-directions ( \ref{['eq:lossf']}).
• Figure 5: Our approach can represent the transparency of a glassy object (Objaverse dataset objaverseXL) achieving 30.6 PSNR whereas MobileNeRF fails with only 24.6 PSNR. To effectively test transparency, the shape is also rendered against a textured background. Please zoom-in to see details.