TAMBRIDGE: Bridging Frame-Centered Tracking and 3D Gaussian Splatting for Enhanced SLAM

TL;DR

TAMBRIDGE addresses the fragility and non-real-time performance of 3D Gaussian Splatting (3DGS) SLAM in noisy, motion-blurred, long-session settings by introducing a plug-and-play Fusion Bridge that fuses frame-centered tracking (ORB Visual Odometry) with mapping-centered online 3DGS. The approach jointly optimizes reprojection, color, and depth rendering losses while leveraging viewpoint covisibility to select informative views, yielding robust pose initialization and faster convergence. The method demonstrates state-of-the-art rendering quality and localization accuracy on real-world datasets, achieving near-real-time performance (>5 FPS) and significantly improved robustness to noise and blur compared with NeRF-based SLAM and prior 3DGS systems. By enabling a stable, near-real-time SLAM workflow in challenging robotic scenarios, TAMBRIDGE offers a practical solution for real-world robotics requiring reliable environmental understanding and fast mapping updates.

Abstract

The limited robustness of 3D Gaussian Splatting (3DGS) to motion blur and camera noise, along with its poor real-time performance, restricts its application in robotic SLAM tasks. Upon analysis, the primary causes of these issues are the density of views with motion blur and the cumulative errors in dense pose estimation from calculating losses based on noisy original images and rendering results, which increase the difficulty of 3DGS rendering convergence. Thus, a cutting-edge 3DGS-based SLAM system is introduced, leveraging the efficiency and flexibility of 3DGS to achieve real-time performance while remaining robust against sensor noise, motion blur, and the challenges posed by long-session SLAM. Central to this approach is the Fusion Bridge module, which seamlessly integrates tracking-centered ORB Visual Odometry with mapping-centered online 3DGS. Precise pose initialization is enabled by this module through joint optimization of re-projection and rendering loss, as well as strategic view selection, enhancing rendering convergence in large-scale scenes. Extensive experiments demonstrate state-of-the-art rendering quality and localization accuracy, positioning this system as a promising solution for real-world robotics applications that require stable, near-real-time performance. Our project is available at https://ZeldaFromHeaven.github.io/TAMBRIDGE/

Figures (11)

• Figure 1: By seamlessly integrating ORB Visual Odometry with viewpoint selection and re-projection loss, our method significantly improves the robustness towards sensor noise and motion blur especially in long-session robotic tasks.
• Figure 2: Overview. ORB-based visual odometry analyzes RGB-D data to estimate poses and select keyframes for local mapping. These keyframes feed into the Global Optimization featuring Bundle Adjustment (BA) to refine the path. Simultaneously, the Fusion Bridge module selects reconstruction frames and calculates rendering poses and Border Masks. An online 3DGS backend processes the selected frames to create a globally consistent, high-fidelity scene representation.
• Figure 3: The Fusion Bridge module selects reconstruction keyframes from a local map based on viewpoint covisibility. It projects the 3DGS and local map point cloud onto the reconstruction frame, filtering projections through pixel gates. The module then optimizes the pose by jointly minimizing rendering and point cloud reprojection losses, setting the initial rendering pose for the Online 3DGS.
• Figure 4: Qualitative results on TUM RGBD fr1_desk and fr3_office sequences.
• Figure 5: (a) The change in TAMBRIDGE PSNR with FPS. (b) The change in TAMBRIDGE SSIM with FPS. (c) The change in TAMBRIDGE LPIPS with FPS.