Frequency-Aware Gaussian Splatting Decomposition

TL;DR

The paper targets the limitation of 3D Gaussian Splatting (3D-GS) which treats all frequencies uniformly by introducing a frequency-aware decomposition that groups Gaussians into Laplacian-pyramid subbands of input image frequencies. A progressive coarse-to-fine training scheme with a frequency regularizer and signed residual colors enforces spectral separation and enables per-band control, while retaining full-resolution rendering to avoid aliasing. The approach achieves state-of-the-art reconstruction quality and rendering speed among LOD-capable baselines and unlocks practical applications including level-of-detail streaming, foveated rendering, promptable 3D focus, and artistic 3D filters. This work advances both interpretability and usability of 3D scene representations, enabling efficient, frequency-aware rendering and interactive content manipulation in real time.

Abstract

3D Gaussian Splatting (3D-GS) enables efficient novel view synthesis, but treats all frequencies uniformly, making it difficult to separate coarse structure from fine detail. Recent works have started to exploit frequency signals, but lack explicit frequency decomposition of the 3D representation itself. We propose a frequency-aware decomposition that organizes 3D Gaussians into groups corresponding to Laplacian-pyramid subbands of the input images. Each group is trained with spatial frequency regularization to confine it to its target frequency, while higher-frequency bands use signed residual colors to capture fine details that may be missed by lower-frequency reconstructions. A progressive coarse-to-fine training schedule stabilizes the decomposition. Our method achieves state-of-the-art reconstruction quality and rendering speed among all LOD-capable methods. In addition to improved interpretability, our method enables dynamic level-of-detail rendering, progressive streaming, foveated rendering, promptable 3D focus, and artistic filtering. Our code will be made publicly available.

Figures (14)

• Figure 1: Frequency-Aware Decomposition for 3D Gaussian Splatting: We introduce a frequency-aware decomposition for 3D Gaussian Splatting (3D-GS). The decomposition assigns different Gaussians to different levels of the Laplacian Pyramid of the input images. This enables Level of Detail (LOD) rendering, Promptable 3D focus, Foveated rendering, and 3D artistic filters. Our method achieves the highest PSNR, SSIM, and rendering speed among all LOD-capable baselines.
• Figure 2: Frequency-Aware 3D Gaussian Splatting Overview: We decompose 3D Gaussians into frequency-specific groups to construct a multi-scale representation. (Left) Each Gaussian group targets a specific frequency band: Level 1 captures low-frequency structure, while Levels 2-3 add mid and high-frequency details through signed residual colors. (Middle) During training, each accumulated level (L1, L1+L2, L1+L2+L3) is supervised against appropriately blurred ground truth images, with red dashed lines showing gradient flow that constrains lower levels to remain frequency-limited while higher levels capture residual details. (Right) Our dual loss combines standard pixel loss with DFT magnitude loss in frequency space, ensuring each level contributes meaningful frequency content and preventing frequency leakage between bands.
• Figure 3: Visualization of the Gaussian frequency levels distribution in the scene. The colormap represents different frequency levels, where lower levels correspond to coarse scene structures and higher levels capture finer details.
• Figure 4: Progressive Rendering Consistency: Comparing between OCTree-GS (top) and our method (bottom) across different frequency levels (left to right). In OCTree-GS, lower levels suffer from missing geometric details, which are recovered as more levels are added. In contrast, our representation preserves the scene’s geometry across all levels, with fine details progressively added, making it more suitable for frequency-aware applications.
• Figure 5: Progressive Rendering and Streaming: Comparison of our method, FLOD, and OCTree-GS across different levels of detail. The plot shows the trade-off between rendering speed (FPS) and reconstruction quality (PSNR), where each point corresponds to a specific LOD level and bubble size reflects the number of Gaussians. Results are averaged over all Mip-NeRF 360 scenes. Our method demonstrating superior performance in both quality and speed in the highest three LODs.