F-3DGS: Factorized Coordinates and Representations for 3D Gaussian Splatting

TL;DR

This work tackles the storage bottleneck of 3D Gaussian Splatting (3DGS) used in fast neural rendering by introducing Factorized 3D Gaussian Splatting (F-3DGS). It employs two factorization schemes, canonical polyadic (CP) and vector-matrix (VM), to compress coordinates and per-Gaussian attributes, complemented by trainable binary masking to prune non-contributing Gaussians. The approach achieves substantial storage reductions (often >90%) while preserving image quality and enabling real-time rendering across multiple datasets, including synthetic-NeRF, Tanks&Temples, and Mip-NeRF360. By avoiding fixed grids and leveraging factorized representations, F-3DGS generalizes to large or unbounded scenes with scalable performance, marking a practical advancement for resource-constrained neural rendering applications.

Abstract

The neural radiance field (NeRF) has made significant strides in representing 3D scenes and synthesizing novel views. Despite its advancements, the high computational costs of NeRF have posed challenges for its deployment in resource-constrained environments and real-time applications. As an alternative to NeRF-like neural rendering methods, 3D Gaussian Splatting (3DGS) offers rapid rendering speeds while maintaining excellent image quality. However, as it represents objects and scenes using a myriad of Gaussians, it requires substantial storage to achieve high-quality representation. To mitigate the storage overhead, we propose Factorized 3D Gaussian Splatting (F-3DGS), a novel approach that drastically reduces storage requirements while preserving image quality. Inspired by classical matrix and tensor factorization techniques, our method represents and approximates dense clusters of Gaussians with significantly fewer Gaussians through efficient factorization. We aim to efficiently represent dense 3D Gaussians by approximating them with a limited amount of information for each axis and their combinations. This method allows us to encode a substantially large number of Gaussians along with their essential attributes -- such as color, scale, and rotation -- necessary for rendering using a relatively small number of elements. Extensive experimental results demonstrate that F-3DGS achieves a significant reduction in storage costs while maintaining comparable quality in rendered images.

Figures (8)

• Figure 1: Examples of factorized coordinates: (a) 25 normal coordinates, (b) 5 $\times$ 5 factorized coordinates. each $x$ and $y$ axis has 5 points, and both represent 25 (5 $\times$ 5) points. (c) two 5 $\times$ 5 factorized coordinates and a total of 50 points are represented (2 $\times$ 5 $\times$ 5), (d) multi-resolution factorized coordinates, where two factorized coordinates have different resolutions (3 $\times$ 3 and 5 $\times$ 5), represent total 34 points, (e) two 3 $\times$ 3 and one 5 $\times$ 5 factorized coordinates. A total of 43 points are represented. The best-viewed in color.
• Figure 2: Illustration of factorized coordinates and representations. $p$, $s$, $q$, and $f$ denote coordinate, scale, rotation (in quaternion), and features for color and opacities, respectively. The lower indices of $s$, $q$, and $f$ are the axis and the indices of the feature dimension. For element-wise multiplication, we used the $\odot$ notation.
• Figure 3: Visualization of factorized coordinate sets. The right figure shows the approximation of 2,488 three-dimensional coordinates using only 30 factorized coordinate sets with an $N_b$ of three.
• Figure 4: Visualiztion of F-3DGS and 3DGS. These visualize Gaussian points, ellipsoids, and rendered images of six objects. We present the storage requirements for our CP-16 F-3DGS.
• Figure 5: Qualitative results. For our method, we used CP with a $d$ of 16 in the case of our model, which is about 4–7 MB. For TensoRF, we visualized VM-48, which is about 16 MB. For 3DGS, we used the original 3DGS of 40–50 MB.