Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing 3D Gaussian Splatting for Sparse Viewpoint Scene Reconstruction

Shen Chen, Jiale Zhou, Lei Li

TL;DR

SVS-GS, a novel framework for Sparse Viewpoint Scene reconstruction that integrates a 3D Gaussian smoothing filter to suppress artifacts, is introduced, offering a robust and efficient solution for scene understanding in robotics and computer vision applications.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a promising approach for 3D scene representation, offering a reduction in computational overhead compared to Neural Radiance Fields (NeRF). However, 3DGS is susceptible to high-frequency artifacts and demonstrates suboptimal performance under sparse viewpoint conditions, thereby limiting its applicability in robotics and computer vision. To address these limitations, we introduce SVS-GS, a novel framework for Sparse Viewpoint Scene reconstruction that integrates a 3D Gaussian smoothing filter to suppress artifacts. Furthermore, our approach incorporates a Depth Gradient Profile Prior (DGPP) loss with a dynamic depth mask to sharpen edges and 2D diffusion with Score Distillation Sampling (SDS) loss to enhance geometric consistency in novel view synthesis. Experimental evaluations on the MipNeRF-360 and SeaThru-NeRF datasets demonstrate that SVS-GS markedly improves 3D reconstruction from sparse viewpoints, offering a robust and efficient solution for scene understanding in robotics and computer vision applications.

Optimizing 3D Gaussian Splatting for Sparse Viewpoint Scene Reconstruction

TL;DR

SVS-GS, a novel framework for Sparse Viewpoint Scene reconstruction that integrates a 3D Gaussian smoothing filter to suppress artifacts, is introduced, offering a robust and efficient solution for scene understanding in robotics and computer vision applications.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a promising approach for 3D scene representation, offering a reduction in computational overhead compared to Neural Radiance Fields (NeRF). However, 3DGS is susceptible to high-frequency artifacts and demonstrates suboptimal performance under sparse viewpoint conditions, thereby limiting its applicability in robotics and computer vision. To address these limitations, we introduce SVS-GS, a novel framework for Sparse Viewpoint Scene reconstruction that integrates a 3D Gaussian smoothing filter to suppress artifacts. Furthermore, our approach incorporates a Depth Gradient Profile Prior (DGPP) loss with a dynamic depth mask to sharpen edges and 2D diffusion with Score Distillation Sampling (SDS) loss to enhance geometric consistency in novel view synthesis. Experimental evaluations on the MipNeRF-360 and SeaThru-NeRF datasets demonstrate that SVS-GS markedly improves 3D reconstruction from sparse viewpoints, offering a robust and efficient solution for scene understanding in robotics and computer vision applications.
Paper Structure (20 sections, 19 equations, 4 figures, 2 tables)

This paper contains 20 sections, 19 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Novel View Synthesis
  3. Primitive-Based Rendering
  4. Preliminaries
  5. Methods
  6. Problem Formulation
  7. Initialize Adaptive Dense
  8. Global Processing
  9. Local Processing
  10. Density-weighted selection
  11. 3D Smoothing
  12. Depth SDS as Optimization Guidance
  13. Depth mask and Gradient Profile Prior
  14. Loss Function
  15. Experiments
  16. ...and 5 more sections

Figures (4)

  • Figure 1: We propose a sparse Viewpoint scene Reconstruction framework. Comparison of 3DGS kerbl20233d and SparseGS xiong2023sparsegs with our SVS-GS trained on 8 views shows that SVS-GS outperforms the other methods in synthesizing close-up scenes.
  • Figure 2: Overall framework. Starting with multi-view images and corresponding depth maps (obtained from Monocular Depth Estimation Models miangoleh2021boosting), point clouds are generated by SfM and undergo adaptive density processing to optimize the density distribution of the point clouds. The point clouds are initialized as 3D Gaussian distributions and further refined through operations such as RGB, depth, and DGPP Loss. The SDS loss function is integrated to ensure geometric consistency and reduce noise.
  • Figure 3: Qualitative results on the Mip-NeRF 360 dataset show that our approach is perceptually similar to the ground truth.
  • Figure 4: Qualitative results on the SeaThru-NeRF dataset show that our method can effectively shield the influence of distant scenery.