ConeGS: Error-Guided Densification Using Pixel Cones for Improved Reconstruction with Fewer Primitives

TL;DR

ConeGS addresses suboptimal primitive distribution in 3D Gaussian Splatting by replacing cloning-based densification with an image-space, error-guided densification that places pixel-cone sized Gaussians along iNGP-predicted depths. By initializing new primitives from pixel footprints and controlling growth with pre-activation opacity penalties and two budgeting strategies, ConeGS delivers higher reconstruction quality and faster rendering across budgets, especially under tight primitive limits. The approach maintains compatibility with other 3DGS improvements and reduces reliance on existing scene geometry, enabling more uniform coverage and efficient optimization. Overall, ConeGS demonstrates robust gains in fidelity and speed, with practical impact for real-time view synthesis under resource constraints.

Abstract

3D Gaussian Splatting (3DGS) achieves state-of-the-art image quality and real-time performance in novel view synthesis but often suffers from a suboptimal spatial distribution of primitives. This issue stems from cloning-based densification, which propagates Gaussians along existing geometry, limiting exploration and requiring many primitives to adequately cover the scene. We present ConeGS, an image-space-informed densification framework that is independent of existing scene geometry state. ConeGS first creates a fast Instant Neural Graphics Primitives (iNGP) reconstruction as a geometric proxy to estimate per-pixel depth. During the subsequent 3DGS optimization, it identifies high-error pixels and inserts new Gaussians along the corresponding viewing cones at the predicted depth values, initializing their size according to the cone diameter. A pre-activation opacity penalty rapidly removes redundant Gaussians, while a primitive budgeting strategy controls the total number of primitives, either by a fixed budget or by adapting to scene complexity, ensuring high reconstruction quality. Experiments show that ConeGS consistently enhances reconstruction quality and rendering performance across Gaussian budgets, with especially strong gains under tight primitive constraints where efficient placement is crucial.

Figures (14)

• Figure 1: ConeGS replaces cloning-based densification with a novel method that generates pixel-cone-sized primitives in regions of high image-space error. By improving placement and removing reliance on existing scene structure thanks to a flexible iNGP-based exploration, it achieves higher reconstruction quality than baselines using the same number of primitives. Results are averaged over Mip-NeRF 360 barron2022mipnerf360 and OMMO lu2023largescaleoutdoormultimodaldataset datasets, with a visual comparison on the truck scene from Tanks & Temples Knapitsch2017.
• Figure 2: Densification comparison. Cloning-based methods are difficult to tune, and the resulting primitives may require many iterations to fit correctly into the scene. ConeGS, by contrast, places primitives precisely using the pixel viewing cone size, enabling faster scene integration without reliance on the existing geometry.
• Figure 3: Overview of the ConeGS pipeline. ($a$) First, an iNGP reconstruction is obtained to serve as a geometric proxy for object surfaces, guiding the placement of Gaussians both during scene initialization and throughout the 3DGS optimization process. ($b$) During 3DGS optimization, ConeGS performs error-guided densification by sampling a subset of pixels with high $L_1$ error. For each sampled pixel, a new Gaussian $\mathcal{G}$ is created along the pixel's viewing cone at the depth estimated by iNGP and scaled to match the cone's size. New Gaussians are accumulated and, every 100 iterations, inserted into the scene after pruning those with low opacity. Blue arrows indicate gradient updates to Gaussian parameters, and the red arrow marks scene updates.
• Figure 4: Densification overview. Illustration of the proposed error-guided strategy. We render an image with 3DGS, compute the per-pixel $L_1$ error, sample pixels proportionally to their error magnitude, and place new Gaussians at the iNGP-predicted depth along the corresponding viewing rays.
• Figure 5: Qualitative results comparing our method with MCMC kheradmand20243d (with SfM point cloud initialization) and EDGS Kotovenko2025ARXIV_EDGS_Eliminating_Densification on the Mip-NeRF 360 barron2022mipnerf360, OMMO lu2023largescaleoutdoormultimodaldataset, and DeepBlending DeepBlending2018 datasets, with varying Gaussian budgets (given in parentheses).