Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS

Sharath Girish, Kamal Gupta, Abhinav Shrivastava

TL;DR

This work presents a technique utilizing quantized embeddings to significantly reduce per-point memory storage requirements and a coarse-to-fine training strategy for a faster and more stable optimization of the Gaussian point clouds, developing a pruning stage which results in scene representations with fewer Gaussians, leading to faster training times and rendering speeds for real-time rendering of high resolution scenes.

Abstract

Recently, 3D Gaussian splatting (3D-GS) has gained popularity in novel-view scene synthesis. It addresses the challenges of lengthy training times and slow rendering speeds associated with Neural Radiance Fields (NeRFs). Through rapid, differentiable rasterization of 3D Gaussians, 3D-GS achieves real-time rendering and accelerated training. They, however, demand substantial memory resources for both training and storage, as they require millions of Gaussians in their point cloud representation for each scene. We present a technique utilizing quantized embeddings to significantly reduce per-point memory storage requirements and a coarse-to-fine training strategy for a faster and more stable optimization of the Gaussian point clouds. Our approach develops a pruning stage which results in scene representations with fewer Gaussians, leading to faster training times and rendering speeds for real-time rendering of high resolution scenes. We reduce storage memory by more than an order of magnitude all while preserving the reconstruction quality. We validate the effectiveness of our approach on a variety of datasets and scenes preserving the visual quality while consuming 10-20x lesser memory and faster training/inference speed. Project page and code is available https://efficientgaussian.github.io

EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS

TL;DR

This work presents a technique utilizing quantized embeddings to significantly reduce per-point memory storage requirements and a coarse-to-fine training strategy for a faster and more stable optimization of the Gaussian point clouds, developing a pruning stage which results in scene representations with fewer Gaussians, leading to faster training times and rendering speeds for real-time rendering of high resolution scenes.

Abstract

Recently, 3D Gaussian splatting (3D-GS) has gained popularity in novel-view scene synthesis. It addresses the challenges of lengthy training times and slow rendering speeds associated with Neural Radiance Fields (NeRFs). Through rapid, differentiable rasterization of 3D Gaussians, 3D-GS achieves real-time rendering and accelerated training. They, however, demand substantial memory resources for both training and storage, as they require millions of Gaussians in their point cloud representation for each scene. We present a technique utilizing quantized embeddings to significantly reduce per-point memory storage requirements and a coarse-to-fine training strategy for a faster and more stable optimization of the Gaussian point clouds. Our approach develops a pruning stage which results in scene representations with fewer Gaussians, leading to faster training times and rendering speeds for real-time rendering of high resolution scenes. We reduce storage memory by more than an order of magnitude all while preserving the reconstruction quality. We validate the effectiveness of our approach on a variety of datasets and scenes preserving the visual quality while consuming 10-20x lesser memory and faster training/inference speed. Project page and code is available https://efficientgaussian.github.io
Paper Structure (17 sections, 8 equations, 9 figures, 8 tables)

This paper contains 17 sections, 8 equations, 9 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Method
  5. Attribute quantization
  6. Opacity quantization for improved optimization.
  7. Progressive training
  8. Influence Pruning
  9. Experiments
  10. Implementation and evaluation
  11. Benchmark comparison
  12. Ablations
  13. Progressive scaling variants
  14. Training and Rendering Memory
  15. Conclusion
  16. ...and 2 more sections

Figures (9)

  • Figure 1: We propose EAGLES, a technique to significantly reduce the memory footprint of 3D-Gaussian splatting by an order of magnitude while offering further speed ups in training time and rendering FPS all while maintaining the reconstruction quality.
  • Figure 2: Approach: 1) We quantize the attributes of the latents to reduce the storage memory of the Gaussians (\ref{['ssec:quant']}), 2) progressively train using a coarse-to-fine rendering resolution schedule to obtain higher quality reconstructions (\ref{['ssec:quant']} and 3) utilize a pruning stage to obtain fewer Gaussians and faster training/rendering speeds (\ref{['ssec:densify']}).
  • Figure 2: Comparison of our approach on the Deep Blending dataset. We improve reconstruction quality in terms of PSNR compared to 3D-GS while also improving on all efficiency metrics of storage memory, FPS and training time.
  • Figure 3: Left: Histogram of opacity coefficients with and without quantization. Most coefficients result in values of 0 or 1 without quantization while quantization spreads the opacity values and allows for better blending. Right: Opacity gradient visualization for 3D-GS (top) and EAGLES (bottom). We reduce outlier gradients while high positive gradients in 3D-GS saturates erroneous Gaussians which are not pruned.
  • Figure 4: Effect of influence pruning: Our approach identifies Gaussians which do not contribute significantly to the rasterization process and prune them without drop in reconstruction quality.
  • ...and 4 more figures