Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graphical X Splatting (GraphiXS): A Graphical Model for 4D Gaussian Splatting under Uncertainty

Doga Yilmaz, Jialin Zhu, Deshan Gong, He Wang

TL;DR

GraphiXS introduces a probabilistic graphical framework for 4D Gaussian Splatting to explicitly model multi-type data uncertainty, including missing views, temporal sparsity, and camera asynchrony. It formulates a generative process with a mixture of components, enables MAP inference, and treats camera poses and times as latent random variables, with priors and higher-order dynamics to regulate behavior. The framework is instantiated with Gaussian and Student's-t components and can upgrade existing 4DGS methods by adding stochastic components and priors, showing robust performance across diverse uncertainty settings. Experiments on the N3DV dataset demonstrate clear improvements over baselines under standard and uncertain conditions, highlighting GraphiXS's potential to enhance robustness and generalization in neural rendering. The work offers a principled, generalizable path toward uncertainty-aware 4DGS with practical impact for robust 3D reconstruction and neural rendering workflows.

Abstract

We propose a new framework to systematically incorporate data uncertainty in Gaussian Splatting. Being the new paradigm of neural rendering, Gaussian Splatting has been investigated in many applications, with the main effort in extending its representation, improving its optimization process, and accelerating its speed. However, one orthogonal, much needed, but under-explored area is data uncertainty. In standard 4D Gaussian Splatting, data uncertainty can manifest as view sparsity, missing frames, camera asynchronization, etc. So far, there has been little research to holistically incorporating various types of data uncertainty under a single framework. To this end, we propose Graphical X Splatting, or GraphiXS, a new probabilistic framework that considers multiple types of data uncertainty, aiming for a fundamental augmentation of the current 4D Gaussian Splatting paradigm into a probabilistic setting. GraphiXS is general and can be instantiated with a range of primitives, e.g. Gaussians, Student's-t. Furthermore, GraphiXS can be used to `upgrade' existing methods to accommodate data uncertainty. Through exhaustive evaluation and comparison, we demonstrate that GraphiXS can systematically model various uncertainties in data, outperform existing methods in many settings where data are missing or polluted in space and time, and therefore is a major generalization of the current 4D Gaussian Splatting research.

Graphical X Splatting (GraphiXS): A Graphical Model for 4D Gaussian Splatting under Uncertainty

TL;DR

GraphiXS introduces a probabilistic graphical framework for 4D Gaussian Splatting to explicitly model multi-type data uncertainty, including missing views, temporal sparsity, and camera asynchrony. It formulates a generative process with a mixture of components, enables MAP inference, and treats camera poses and times as latent random variables, with priors and higher-order dynamics to regulate behavior. The framework is instantiated with Gaussian and Student's-t components and can upgrade existing 4DGS methods by adding stochastic components and priors, showing robust performance across diverse uncertainty settings. Experiments on the N3DV dataset demonstrate clear improvements over baselines under standard and uncertain conditions, highlighting GraphiXS's potential to enhance robustness and generalization in neural rendering. The work offers a principled, generalizable path toward uncertainty-aware 4DGS with practical impact for robust 3D reconstruction and neural rendering workflows.

Abstract

We propose a new framework to systematically incorporate data uncertainty in Gaussian Splatting. Being the new paradigm of neural rendering, Gaussian Splatting has been investigated in many applications, with the main effort in extending its representation, improving its optimization process, and accelerating its speed. However, one orthogonal, much needed, but under-explored area is data uncertainty. In standard 4D Gaussian Splatting, data uncertainty can manifest as view sparsity, missing frames, camera asynchronization, etc. So far, there has been little research to holistically incorporating various types of data uncertainty under a single framework. To this end, we propose Graphical X Splatting, or GraphiXS, a new probabilistic framework that considers multiple types of data uncertainty, aiming for a fundamental augmentation of the current 4D Gaussian Splatting paradigm into a probabilistic setting. GraphiXS is general and can be instantiated with a range of primitives, e.g. Gaussians, Student's-t. Furthermore, GraphiXS can be used to `upgrade' existing methods to accommodate data uncertainty. Through exhaustive evaluation and comparison, we demonstrate that GraphiXS can systematically model various uncertainties in data, outperform existing methods in many settings where data are missing or polluted in space and time, and therefore is a major generalization of the current 4D Gaussian Splatting research.
Paper Structure (29 sections, 20 equations, 7 figures, 7 tables)

This paper contains 29 sections, 20 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Methods
  4. Learning-based Reconstruction Method
  5. Dynamic Reconstruction
  6. Uncertainty in Reconstruction
  7. Methodology
  8. Preliminaries
  9. 3D/4DGS as a mixture model
  10. Graphical model
  11. GraphiXS as a Graphical Model
  12. Notations
  13. Probabilistic Image Reconstruction
  14. Prior for Component Parameters
  15. Objective Function and Optimization
  16. ...and 14 more sections

Figures (7)

  • Figure 1: A graphical model for GraphiXS. The colors correspond to the 4 steps of the generative process (1: red, 2: green, 3: yellow, 4: purple)
  • Figure 2: Comparison under standard setting. Per-region PSNR scores are given at the top right of each image. Enlarged regions contain complex and fast motions e.g. the tongs motion. GraphiXS reconstruction is more clear and richer in details than other methods.
  • Figure 3: Comparison under 10%, 30%, and 50% spatial sparsity. Per-region PSNR scores are given at the top right of each crop. GraphiXS is affected the least when the percentage of missing cameras increases.
  • Figure 4: Comparison under 20 FPS and 10 FPS temporal sparsity. Per-region PSNR scores are given at the top right of each crop. GraphiXS is affected the least when the training camera FPS drops.
  • Figure 5: Comparison under faulty camera 1 and faulty camera 2 settings. Per-region PSNR scores are given at the top right of each crop. GraphiXS is affected the least when spatio-temporal sparsity is increased.
  • ...and 2 more figures