Proxy-GS: Efficient 3D Gaussian Splatting via Proxy Mesh

TL;DR

Proxy-GS tackles the inefficiency of MLP-based 3D Gaussian Splatting in occluded scenes by introducing a lightweight proxy mesh that provides fast depth maps for occlusion-aware culling and proxy-guided anchor densification. The approach couples a proxy-guided filter with training-time densification to align anchors with visible surfaces, reducing redundant decoding and boosting rendering speed. Extensive experiments across MatrixCity, Zip-NeRF, and real-world datasets show consistent quality gains and a substantial FPS improvement over state-of-the-art baselines, especially in occlusion-rich environments. The method demonstrates strong compatibility with existing 3DGS renderers and offers practical impact for real-time VR/AR, large-scale scene reconstruction, and urban indoor/outdoor applications.

Abstract

3D Gaussian Splatting (3DGS) has emerged as an efficient approach for achieving photorealistic rendering. Recent MLP-based variants further improve visual fidelity but introduce substantial decoding overhead during rendering. To alleviate computation cost, several pruning strategies and level-of-detail (LOD) techniques have been introduced, aiming to effectively reduce the number of Gaussian primitives in large-scale scenes. However, our analysis reveals that significant redundancy still remains due to the lack of occlusion awareness. In this work, we propose Proxy-GS, a novel pipeline that exploits a proxy to introduce Gaussian occlusion awareness from any view. At the core of our approach is a fast proxy system capable of producing precise occlusion depth maps at a resolution of 1000x1000 under 1ms. This proxy serves two roles: first, it guides the culling of anchors and Gaussians to accelerate rendering speed. Second, it guides the densification towards surfaces during training, avoiding inconsistencies in occluded regions, and improving the rendering quality. In heavily occluded scenarios, such as the MatrixCity Streets dataset, Proxy-GS not only equips MLP-based Gaussian splatting with stronger rendering capability but also achieves faster rendering speed. Specifically, it achieves more than 2.5x speedup over Octree-GS, and consistently delivers substantially higher rendering quality. Code will be public upon acceptance.

Figures (6)

• Figure 1: We propose Proxy-GS, an occlusion-aware training and inference framework built upon lightweight proxies. By introducing proxy-guided densification, our method effectively guides anchors to grow in more geometrically meaningful regions. As a result, Proxy-GS not only achieves higher rendering quality but also delivers significantly faster rendering compared to state-of-the-art MLP-based 3DGS approaches.
• Figure 2: Proxy-GS Framework. We first construct a lightweight proxy mesh. During rendering, hardware rasterization produces a depth map in under 1 ms, which is then used to efficiently cull anchors that are occluded. During training, in addition to the same rendering pipeline, we further introduce structure-aware anchor densification, encouraging anchors to grow adaptively along the proxy mesh geometry.
• Figure 3: Comparison of the time proportion of each inference component (Rendering, anchor filter, depth rendering) with that of Octree-GS.
• Figure 4: Qualitative comparison. Visualization on different datasets li2023matrixcityxiong2024gauubarron2023zipkerbl2024hierarchical.
• Figure 5: Visualization on different safety margins.