Effective Gaussian Management for High-fidelity Object Reconstruction

TL;DR

This paper introduces a novel densification strategy called \emph{GauSep} that selectively activates Gaussian color or normal attributes and develops an adaptive and integrated Gaussian representation that reduces redundancy both at the individual and global levels, effectively balancing model capacity and number of parameters.

Abstract

This paper presents an effective Gaussian management framework for high-fidelity scene reconstruction of appearance and geometry. Departing from recent Gaussian Splatting (GS) methods that rely on indiscriminate attribute assignment, our approach introduces a novel densification strategy called \emph{GauSep} that selectively activates Gaussian color or normal attributes. Together with a tailored rendering pipeline, termed \emph{Separate Rendering}, this strategy alleviates gradient conflicts arising from dual supervision and yields improved reconstruction quality. In addition, we develop \emph{GauRep}, an adaptive and integrated Gaussian representation that reduces redundancy both at the individual and global levels, effectively balancing model capacity and number of parameters. To provide reliable geometric supervision essential for effective management, we also introduce \emph{CoRe}, a novel surface reconstruction module that distills normal fields from the SDF branch to the Gaussian branch through a confidence mechanism. Notably, our management framework is model-agnostic and can be seamlessly incorporated into other architectures, simultaneously improving performance and reducing model size. Extensive experiments demonstrate that our approach achieves superior performance in reconstructing both appearance and geometry compared with state-of-the-art methods, while using significantly fewer parameters.

Figures (11)

• Figure 1: Comparative analysis of gradient conflicts induced by different Gaussian-normal definitions. The three columns show: (left) scene-fitting results, (middle) loss-optimization traces, and (right) the inner product between gradients of the RGB image and the normal map with respect to Gaussian attributes. For the inner product, the sign denotes whether RGB- and normal-driven gradients are aligned (positive) or opposed (negative), and the magnitude measures the severity of their disagreement. We evaluate three representative normal-definition strategies: (i) an explicit learnable normal attribute per Gaussian, (ii) alignment with the Gaussian's shortest principal axis, and (iii) deriving the normal from a flattened Gaussian (Surfels). All strategies exhibit gradient reversals during optimization; strategy (i) yields the most pronounced conflicts, whereas strategies (ii) and (iii) show similar patterns due to Gaussian flattening during optimization. These results underline the importance of conflict-aware Gaussian management when jointly supervising RGB and normal signals.
• Figure 2: An illustration of our approach. Given multi-view RGB images, we represent the scene using a collection of Gaussians, which are partitioned into three subsets through the Gaussian Separate Operation: Common Gaussians, Color-active Gaussians, and Normal-active Gaussians. We further introduce GauRep, an adaptive and integrated Gaussian representation, to reduce redundancy both at the individual level (adaptive color representation) and the global level (task-decoupled pruning). Task-decoupled pruning is applied to all Gaussian subsets, while adaptive color representation is specifically employed for those with active color attributes (i.e., Common Gaussians and Color-active Gaussians). To ensure high-quality normal supervision, we further propose CoRe, a surface reconstruction module that distills normal fields from the SDF-based implicit branch $\mathcal{B}_{v}$ to the Gaussian branch $\mathcal{B}_{g}$ through a novel confidence mechanism. Finally, high-fidelity and competitive renderings, including images, normal maps, and depth maps, are produced through Separate Rendering, which is tailored to the three Gaussian subsets. We employ screened Poisson reconstruction to extract surface from the normal and depth maps.
• Figure 3: Effect of SDF Supervision. Without SDF supervision, Gaussian SurfelsDai2024GaussianSurfels erroneously interpret specular highlights as geometric variations, leading to degraded surface quality. Incorporating SDF supervision effectively mitigates this issue and yields more faithful geometry reconstruction.
• Figure 4: Illustration of Gaussian Separate Operation. For each Gaussian, we first compute the inner product of its radiance and geometry gradients. When the product is positive, standard clone and split operations are performed; otherwise, Gaussian Separate Operation applies attribute decoupling and creates separate Gaussians specialized for color or geometry. This results in three disjoint Gaussian sets that are optimized under distinct supervision signals for subsequent rendering.
• Figure 5: Qualitative Comparison of Appearance Reconstruction with SOTA methods on BMVS and DTU Datasets.