NeuralGS: Bridging Neural Fields and 3D Gaussian Splatting for Compact 3D Representations

TL;DR

NeuralGS tackles the storage bottleneck of 3D Gaussian Splatting by encoding Gaussian attributes with a set of cluster-specific tiny MLPs, guided by Gaussian global importance. It adopts an importance-weighted fitting, attribute-based clustering, and a frequency-aware fine-tuning regime to preserve rendering quality while achieving substantial compression. The approach yields up to ~117× size reductions with competitive PSNR/SSIM/LPIPS and faster rendering compared to prior methods, and supports progressive, JPEG-like loading for streaming scenarios. This combination of neural-field encoding and efficient cluster-based fitting offers a practical route to compact, real-time 3D representations suitable for large-scale scenes.

Abstract

3D Gaussian Splatting (3DGS) achieves impressive quality and rendering speed, but with millions of 3D Gaussians and significant storage and transmission costs. In this paper, we aim to develop a simple yet effective method called NeuralGS that compresses the original 3DGS into a compact representation. Our observation is that neural fields like NeRF can represent complex 3D scenes with Multi-Layer Perceptron (MLP) neural networks using only a few megabytes. Thus, NeuralGS effectively adopts the neural field representation to encode the attributes of 3D Gaussians with MLPs, only requiring a small storage size even for a large-scale scene. To achieve this, we adopt a clustering strategy and fit the Gaussians within each cluster using different tiny MLPs, based on importance scores of Gaussians as fitting weights. We experiment on multiple datasets, achieving a 91-times average model size reduction without harming the visual quality.

Figures (6)

• Figure 2: NeuralGS directly compresses original 3DGS with neural fields into a compact and rendering-efficient representation. NeRF-based methods typically require minimal storage with slow rendering speeds while 3D Gaussian Splatting (3DGS) achieves fast rendering but demands hundreds of megabytes storage. NeuralGS combines compact neural fields with 3DGS by encoding 3D Gaussian attributes with neural fields, achieving significant reduction in model size and real-time rendering speed.
• Figure 3: Overview of NeuralGS . (A) In Sec. \ref{['sec:important']}, for each Gaussian $\text{GS}_j$ in the scene, we first calculate its global importance score $S_j$ (Eq.\ref{['eq:significance_score']}) and prune unimportant Gaussians. (B) In Sec. \ref{['sec:clustering']}, we cluster the retained Gaussians and use different tiny MLPs to map the positions to Gaussian attributes of different clusters with the loss (Eq.\ref{['eq:importance_loss']}) using the importance score as weights. (C) In Sec. \ref{['sec:finetune']}, we fine-tune the tiny MLPs of all clusters with photorealistic loss (Eq.\ref{['eq:photorealistic_loss']}) and frequency loss (Eq.\ref{['eq:freq_loss']}) to restore quality.
• Figure 4: Details of Cluster-based Neural Field Fitting. The positions of the 3D Gaussians within each cluster are fed into the corresponding tiny MLP to fit the attributes with the importance-weighted loss. During rendering, the predicted outputs are then split into the respective attributes of the Gaussians, i.e., rotation, scale, opacity, color, and SH coefficients.
• Figure 5: Qualitative results of the proposed method compared to existing compression methods.
• Figure 6: NeuralGS allows progressive loading new clusters in the playroom scene to obtain more details and sharper texture.