N-BVH: Neural ray queries with bounding volume hierarchies

TL;DR

N-BVH introduces a compact neural representation tailored for ray queries, integrated with a bounding volume hierarchy to enable efficient, near-surface probing and strong empty-space skipping. The approach combines a multi-resolution hash grid with a shallow BVH and an error-driven, coarse-to-fine training loop to achieve adaptive level-of-detail. It yields substantial memory compression while preserving key rendering signals such as visibility, depth, and appearance, demonstrated through hybrid path tracing and neural appearance prefiltering applications. The work enables complex scenes to be rendered within memory constraints and opens pathways for hardware-accelerated fused neural-ray tracing in the future.

Abstract

Neural representations have shown spectacular ability to compress complex signals in a fraction of the raw data size. In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures, making them ideal candidates for neural compression. Here, the main challenge lies in finding good trade-offs between efficient compression and cheap inference while minimizing training time. In the context of rendering, we adopt a ray-centric approach to this problem and devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our compact model is learned from the input geometry and substituted for it whenever a ray intersection is queried by a path-tracing engine. While prior neural compression methods have focused on point queries, ours proposes neural ray queries that integrate seamlessly into standard ray-tracing pipelines. At the core of our method, we employ an adaptive BVH-driven probing scheme to optimize the parameters of a multi-resolution hash grid, focusing its neural capacity on the sparse 3D occupancy swept by the original surfaces. As a result, our N-BVH can serve accurate ray queries from a representation that is more than an order of magnitude more compact, providing faithful approximations of visibility, depth, and appearance attributes. The flexibility of our method allows us to combine and overlap neural and non-neural entities within the same 3D scene and extends to appearance level of detail.

Figures (14)

• Figure 1: Our lightweight $\mathcal{N}$-BVH is a shallow hierarchy whose leaf nodes simply store bounds within which we search for ray intersections by querying a neural geometry representation. The leaves (in purple outlines) represent a cut in a classical BVH over the input geometry; we optimize the cut by iteratively splitting the leaves with largest inference error. After each splitting step we train our neural model within the cut-node bounds by sampling random rays in the scene.
• Figure 1: Performance comparison against classical path tracing. Our approach achieves 2--4$\times$ higher render times. However, our representation delivers drastic memory compression at low error, allowing the rendering of complex scenes that could not even be uploaded onto low-end GPUs. Reducing the node count of our $\mathcal{N}$-BVH (${\bullet}\mathllap{\circ}$) closes the gap to software path-tracing performance (albeit at a higher error), showing the flexibility of our approach to adapt to strict render-time budgets.
• Figure 2: Our neural ray query pipeline. We sample uniformly along a given ray-box intersection interval and at each point collect features from a multi-resolution hash grid. The concatenated features are fed to an MLP to obtain a reconstructed signal (visibility, intersection, normal, appearance).
• Figure 3: A surface textured with a square checkerboard, observed orthographically at a 45$^{\circ}$ angle. Reconstruction quality is high when our neural representation is probed close to the surface. Increasing the sampling rate along rays reduces error, but at proportionately higher inference cost.
• Figure 4: Increasing the number of $\mathcal{N}$-BVH leaf nodes consistently improves the reconstruction quality. It also improves performance on this scene with low depth complexity where at most one neural inference per ray is needed. The top row visualizes the average training loss per node in false color.