Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Matryoshka Gaussian Splatting

Zhilin Guo, Boqiao Zhang, Hakan Aktas, Kyle Fogarty, Jeffrey Hu, Nursena Koprucu Aslan, Wenzhao Li, Canberk Baykal, Albert Miao, Josef Bengtson, Chenliang Zhou, Weihao Xia, Cristina Nader Vasconcelos. Cengiz Oztireli

Abstract

The ability to render scenes at adjustable fidelity from a single model, known as level of detail (LoD), is crucial for practical deployment of 3D Gaussian Splatting (3DGS). Existing discrete LoD methods expose only a limited set of operating points, while concurrent continuous LoD approaches enable smoother scaling but often suffer noticeable quality degradation at full capacity, making LoD a costly design decision. We introduce Matryoshka Gaussian Splatting (MGS), a training framework that enables continuous LoD for standard 3DGS pipelines without sacrificing full-capacity rendering quality. MGS learns a single ordered set of Gaussians such that rendering any prefix, the first k splats, produces a coherent reconstruction whose fidelity improves smoothly with increasing budget. Our key idea is stochastic budget training: each iteration samples a random splat budget and optimises both the corresponding prefix and the full set. This strategy requires only two forward passes and introduces no architectural modifications. Experiments across four benchmarks and six baselines show that MGS matches the full-capacity performance of its backbone while enabling a continuous speed-quality trade-off from a single model. Extensive ablations on ordering strategies, training objectives, and model capacity further validate the designs.

Matryoshka Gaussian Splatting

Abstract

The ability to render scenes at adjustable fidelity from a single model, known as level of detail (LoD), is crucial for practical deployment of 3D Gaussian Splatting (3DGS). Existing discrete LoD methods expose only a limited set of operating points, while concurrent continuous LoD approaches enable smoother scaling but often suffer noticeable quality degradation at full capacity, making LoD a costly design decision. We introduce Matryoshka Gaussian Splatting (MGS), a training framework that enables continuous LoD for standard 3DGS pipelines without sacrificing full-capacity rendering quality. MGS learns a single ordered set of Gaussians such that rendering any prefix, the first k splats, produces a coherent reconstruction whose fidelity improves smoothly with increasing budget. Our key idea is stochastic budget training: each iteration samples a random splat budget and optimises both the corresponding prefix and the full set. This strategy requires only two forward passes and introduces no architectural modifications. Experiments across four benchmarks and six baselines show that MGS matches the full-capacity performance of its backbone while enabling a continuous speed-quality trade-off from a single model. Extensive ablations on ordering strategies, training objectives, and model capacity further validate the designs.
Paper Structure (30 sections, 9 equations, 7 figures, 2 tables)

This paper contains 30 sections, 9 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. 3D Gaussian Splatting
  4. Level-of-Detail Rendering
  5. Nested (Matryoshka) Representations
  6. Method
  7. Preliminaries on 3DGS
  8. Matryoshka Gaussian Splatting
  9. Importance Score.
  10. Nested Primitive Representation.
  11. Stochastic Budget Training
  12. Budget Sampling.
  13. Training Objective.
  14. Dynamic Reordering.
  15. Experiments
  16. ...and 15 more sections

Figures (7)

  • Figure 1: Continuous LoD need not sacrifice full-capacity quality to enable budget trade-off. Our method, MGS (top), learns an ordered set of Gaussian primitives whose prefixes yield coherent reconstructions at any splat budget. Compared to CLoD-3DGS CLoD-3DGS-milef2025learning (mid) and CLoD-GS CLoD-GS-cheng2025clod (bot), MGS achieves the highest fidelity at every operating point with quality degrading gracefully under budget reduction. Scene: GardenMipNeRF360-barron2022mip.
  • Figure 2: Framework of Matryoshka Gaussian Splatting.
  • Figure 3: Quality-budget trade-off on Mip-NeRF 360 MipNeRF360-barron2022mip, averaged across all nine scenes. Left: Quality ($\bar{Q}$, Eq. \ref{['eq:mixed_quality']}) vs. FPS. Right: Quality vs. number of Gaussian splats. Curves trace continuous LoD models across prefix ratios 1%--100%, and trace discrete LoD models at their recommended operating points respectively. MGS (ours, dark blue) achieves the highest quality at every speed and splat budget, while spanning a much wider FPS range than any baseline.
  • Figure 4: Qualitative comparison of continuous LoD methods across four benchmarks. Renderings are shown at 5%, 10%, 30%, 60%, and 100% of the full splat budget. We compare MGS with CLoD-3DGS CLoD-3DGS-milef2025learning and CLoD-GS CLoD-GS-cheng2025clod. Under highly constrained budgets (5–10%), MGS maintains coherent reconstructions with PSNR of 21--28 dB, while both baselines suffer from severe artifacts and quality collapse (11--17 dB).
  • Figure 5: Additional qualitative results including a failure case. On stump, train, and rome, MGS maintains higher fidelity across all budget levels. On DrJohnson (bottom), CLoD-3DGS CLoD-3DGS-milef2025learning achieves higher peak PSNR at 100% budget (29.1 vs. 27.7 dB); however, MGS still degrades more gracefully at reduced budgets (5--30%).
  • ...and 2 more figures