RU4D-SLAM: Reweighting Uncertainty in Gaussian Splatting SLAM for 4D Scene Reconstruction

TL;DR

This work proposes a robust and efficient framework, namely Reweighting Uncertainty in Gaussian Splatting SLAM (RU4D-SLAM) for 4D scene reconstruction, that introduces temporal factors into spatial 3D representation while incorporating uncertainty-aware perception of scene changes, blurred image synthesis, and dynamic scene reconstruction.

Abstract

Combining 3D Gaussian splatting with Simultaneous Localization and Mapping (SLAM) has gained popularity as it enables continuous 3D environment reconstruction during motion. However, existing methods struggle in dynamic environments, particularly moving objects complicate 3D reconstruction and, in turn, hinder reliable tracking. The emergence of 4D reconstruction, especially 4D Gaussian splatting, offers a promising direction for addressing these challenges, yet its potential for 4D-aware SLAM remains largely underexplored. Along this direction, we propose a robust and efficient framework, namely Reweighting Uncertainty in Gaussian Splatting SLAM (RU4D-SLAM) for 4D scene reconstruction, that introduces temporal factors into spatial 3D representation while incorporating uncertainty-aware perception of scene changes, blurred image synthesis, and dynamic scene reconstruction. We enhance dynamic scene representation by integrating motion blur rendering, and improve uncertainty-aware tracking by extending per-pixel uncertainty modeling, which is originally designed for static scenarios, to handle blurred images. Furthermore, we propose a semantic-guided reweighting mechanism for per-pixel uncertainty estimation in dynamic scenes, and introduce a learnable opacity weight to support adaptive 4D mapping. Extensive experiments on standard benchmarks demonstrate that our method substantially outperforms state-of-the-art approaches in both trajectory accuracy and 4D scene reconstruction, particularly in dynamic environments with moving objects and low-quality inputs. Code available: https://ru4d-slam.github.io

Figures (5)

• Figure 1: 4D scene reconstruction with RU4D-SLAM. The left side shows the 4D Gaussian map reconstructed by RU4D-SLAM on the Bonn dataset bonn_rgbd_dynamic_dataset, featuring novel synthesized views that capture the temporal motions (top) and a comparison between the RGB input and RU4D-SLAM rendering at the same pose (bottom). On the right, we compare rendered results from MonoGS matsuki2024gaussian, 4DGS-SLAM li20254d, and RU4D-SLAM in dynamic scenes. The numbers at the bottom-right of each image denote the PSNR values (higher is better).
• Figure 2: Overview of RU4D-SLAM. RU4D-SLAM operates in three stages: pose estimation, deformation field initialization, and 4D mapping, all of which are closely linked to the uncertainty map $\boldsymbol{\beta}^2$. In the pose estimation stage, $\boldsymbol{\beta}^2$ supports uncertainty-aware DBA tracking. Before 4D mapping, the uncertainty map is combined with SAM to form RUM, within which deformation nodes are initialized as local motion anchors by a pretrained SpaTracker model. In 4D mapping, node trajectories are propagated to Gaussians via deformation blending and optimized through IR- and AOW-guided training for joint static and dynamic rendering at each keyframe. Snowflake icons denote pre-trained, frozen modules.
• Figure 3: Impact of IR on predicted uncertainty map $\boldsymbol{\beta}^2$. Predicted uncertainty on a sample frame from w_x and w_r sequence in TUM dataset. Higher values are shown in red, lower values in blue, with intermediate values transitioning smoothly.
• Figure 4: Comparison of dynamic object tracking across masks. Performance of YOLO-based mask li20254d, flow-based mask lei2025mosca, and our RUM on three sample frame sets from w_x in TUM, pb1 in Bonn and park in Wild-SLAM.
• Figure 5: Impact of AOW on rendering. Comparison of renderings without and with AOW on frames from w_x in TUM, pb1 in Bonn and park in Wild-SLAM.