Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting

TL;DR

This work diagnoses a prevalent failure mode in 3D Gaussian Splatting (3DGS) where Gaussians collapse into needle-like shapes during optimization. It introduces a differentiable effective rank regularization that discourages $\text{erank}(\mathcal{G}_k) \approx 1$ and promotes disk-like coverage, improving normals and geometry while remaining usable as an add-on to existing 3DGS variants. An improved Adaptive Density Control (ADC) densification scheme complements the regularizer, enabling robust surface reconstruction and more efficient representations. Empirical results on DTU and Mip-NeRF360 show consistent gains in geometry and novel-view synthesis with reduced memory requirements, demonstrating practical benefits for real-time 3D reconstruction and rendering.

Abstract

3D reconstruction from multi-view images is one of the fundamental challenges in computer vision and graphics. Recently, 3D Gaussian Splatting (3DGS) has emerged as a promising technique capable of real-time rendering with high-quality 3D reconstruction. This method utilizes 3D Gaussian representation and tile-based splatting techniques, bypassing the expensive neural field querying. Despite its potential, 3DGS encounters challenges such as needle-like artifacts, suboptimal geometries, and inaccurate normals caused by the Gaussians converging into anisotropic shapes with one dominant variance. We propose using the effective rank analysis to examine the shape statistics of 3D Gaussian primitives, and identify the Gaussians indeed converge into needle-like shapes with the effective rank 1. To address this, we introduce the effective rank as a regularization, which constrains the structure of the Gaussians. Our new regularization method enhances normal and geometry reconstruction while reducing needle-like artifacts. The approach can be integrated as an add-on module to other 3DGS variants, improving their quality without compromising visual fidelity. The project page is available at https://junhahyung.github.io/erankgs.github.io.

Figures (11)

• Figure 1: (a) Qualitative results on novel view synthesis and normal reconstruction on the DTU jensen2014large dataset. (b) and (c) show novel view synthesis comparisons on the Mip-NeRF360 barron2022mip and DTU datasets, respectively. The top row shows novel view renderings of 3DGS, and the bottom row shows renderings of 3DGS with effective rank regularization. While naive 3DGS presents needle-like artifacts, our regularization term mitigates these artifacts in novel views.
• Figure 2: (Green): The effective rank histograms for baseline methods 3DGS kerbl20233d, SuGaR guedon2023sugar, and 2DGS huang20242d, showing that Gaussian ranks are not optimally constrained for geometry reconstruction. (Purple): The regularization term properly constrains the Gaussians, flattening them while preventing convergence into needle-like shapes.
• Figure 3: Real-scale visualization of a 3D sphere and 2D disks and their effective ranks.
• Figure 4: Visualization of the reconstructed mesh using TSDF. Baseline methods often exhibit empty holes, while our regularization term enforces disk-like Gaussians, reducing such artifacts and improving surface reconstruction.
• Figure 5: Normal reconstruction results on the DTU dataset. Needle-like Gaussians often leave empty holes or transparent regions, resulting in hollow or incomplete reconstructions, as seen on the pear surface. The effective rank regularization significantly mitigates these artifacts, leading to more accurate geometry reconstruction.

Theorems & Definitions (1)

• Definition 1: Effective rank