NVGS: Neural Visibility for Occlusion Culling in 3D Gaussian Splatting

TL;DR

NVGS addresses the challenge of occlusion culling in 3D Gaussian Splatting by learning a viewer-dependent visibility function for Gaussians with a compact shared MLP. The pipeline extracts per-Gaussian visibility from trained assets, distills it into an MLP, and uses an occlusion-aware instanced rasterizer to prune Gaussians before instantiation, leveraging Tensor Cores for speed. The approach yields large VRAM reductions and high image quality, enabling real-time rendering of scenes with up to $10^8$ Gaussians and complementing existing LoD techniques. This has practical impact for scalable, high-fidelity rendering in games and film, reducing preprocessing and memory bottlenecks in large-scale composed scenes.

Abstract

3D Gaussian Splatting can exploit frustum culling and level-of-detail strategies to accelerate rendering of scenes containing a large number of primitives. However, the semi-transparent nature of Gaussians prevents the application of another highly effective technique: occlusion culling. We address this limitation by proposing a novel method to learn the viewpoint-dependent visibility function of all Gaussians in a trained model using a small, shared MLP across instances of an asset in a scene. By querying it for Gaussians within the viewing frustum prior to rasterization, our method can discard occluded primitives during rendering. Leveraging Tensor Cores for efficient computation, we integrate these neural queries directly into a novel instanced software rasterizer. Our approach outperforms the current state of the art for composed scenes in terms of VRAM usage and image quality, utilizing a combination of our instanced rasterizer and occlusion culling MLP, and exhibits complementary properties to existing LoD techniques.

Figures (13)

• Figure 1: Shows a scene composed of multiple, separately trained 3DGS assets. Top left shows the gsplat implementation of 3DGSye2025gsplat, Top right V3DG Yang2025V3DG and Bottom Ours. Our approach uses significantly less VRAM, consistently achieves higher image quality, and increases FPS. We achieve this by combining a novel instanced rasterizer and our neural visibility MLP, which enables occlusion culling.
• Figure 2: Left shows the donut asset from the front. Right we show the donut as if rendered from the front from a different viewpoint. We then show the Gaussians that our approach culls in red, along with what we would optimally render.
• Figure 3: Left: Avatar rendered at a near and far distance. Right: We show the original render, then the Gaussians that are not used at the far distance marked in red, and on the right the Gaussians that get used at the far distance.
• Figure 4: We extract per-Gaussian visibility by rendering to a set of training views and baking the visibility information into a lightweight MLP. Per-Gaussian parameters are encoded using a secondary MLP and precomputed once prior to rendering. At render time, our instanced rasterizer renders composed scenes containing both neural-visibility and standard 3DGS assets, leveraging neural visibility whenever available to improve efficiency by culling zero-contribution Gaussians.
• Figure 5: Shows GT render using gsplat, V3DG, and our method for close, medium, and far distances. For V3DG and ours, we show the prediction on the left and FLIP on the right. We scaled FLIP by a factor of five to facilitate better visual comparison.