Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis

Kaiqiang Xiong, Zhanke Wang, Ronggang Wang

TL;DR

This work presents multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis and proposes a geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature pruning.

Abstract

We present multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis. Our sampler fuses complementary cues { -- } photometric rendering residuals, semantic priors, and geometric priors { -- } to produce a robust, local recoverability estimate that directly drives where to inject fine Gaussians. Built around this sampling core, our framework comprises (1) a coarse-to-fine Gaussian representation that encodes global shape with a stable coarse layer and selectively adds fine primitives where the multimodal metric indicates recoverable detail; and (2) a geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature pruning. By prioritizing regions supported by consistent multimodal evidence rather than raw residuals alone, our method alleviates overfitting texture-induced errors and suppresses noise from pose/appearance inconsistencies. Experiments on diverse sparse-view benchmarks demonstrate state-of-the-art reconstructions, with up to +0.3 dB PSNR on DTU.

Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis

TL;DR

This work presents multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis and proposes a geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature pruning.

Abstract

We present multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis. Our sampler fuses complementary cues { -- } photometric rendering residuals, semantic priors, and geometric priors { -- } to produce a robust, local recoverability estimate that directly drives where to inject fine Gaussians. Built around this sampling core, our framework comprises (1) a coarse-to-fine Gaussian representation that encodes global shape with a stable coarse layer and selectively adds fine primitives where the multimodal metric indicates recoverable detail; and (2) a geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature pruning. By prioritizing regions supported by consistent multimodal evidence rather than raw residuals alone, our method alleviates overfitting texture-induced errors and suppresses noise from pose/appearance inconsistencies. Experiments on diverse sparse-view benchmarks demonstrate state-of-the-art reconstructions, with up to +0.3 dB PSNR on DTU.
Paper Structure (13 sections, 5 equations, 3 figures, 2 tables)

This paper contains 13 sections, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminaries
  4. Hierarchical Gaussian Representation
  5. Multi-Modal Importance Assessment
  6. Geometric-Aware Sampling
  7. Training Strategy
  8. Experiments
  9. Experimental Setup
  10. Quantitative Results
  11. Ablation Studies
  12. Qualitative Results
  13. Conclusion

Figures (3)

  • Figure 1: Qualitative comparison of Sparse-view Novel View Synthesis quality with the SOTA methods CoR-GS zhang2024cor and NexusGS zheng2025nexusgs on DTU jensen2014large dataset(3 views). Our method renders more accurate detailed textures.
  • Figure 2: Framework. Our hierarchical Gaussian Splatting framework with Multimodal-Prior-Guided importance sampling for sparse-view novel view synthesis. It consists of three main components: (a) a Hierarchical Gaussian representation with coarse and fine levels (in \ref{['subsec:hier']}), (b) a Multi-Modal Importance Assessment module (in \ref{['subsec:mmia']}), and (c) a Geometric-aware Sampling and pruning strategy (in \ref{['subsec:caas']}).
  • Figure 3: Qualitative results on LLFF mildenhall2019local and DTU jensen2014large Datasets.