VAR-SLAM: Visual Adaptive and Robust SLAM for Dynamic Environments

TL;DR

VAR-SLAM tackles dynamic-environment visual SLAM by fusing lightweight semantic filtering of known moving objects with Barron’s online adaptive robust kernel to downweight unknown dynamics, integrated into ORB-SLAM3. The method updates the loss shape parameter $oldsymbol{ ext{alpha}}$ during Local BA, enabling automatic shifts between near-Gaussian and heavy-tailed residual handling while keeping tracking stable with a fixed $oldsymbol{ ext{alpha}}$ in tracking. Empirical results on TUM, Bonn Dynamic, and OpenLORIS show improved robustness and competitive accuracy, with up to 25% reduction in ATE RMSE on challenging sequences and around 27 FPS on average. The work demonstrates that combining semantic filtering with online adaptive robustness offers a practical, threshold-free path for dynamic-scene SLAM, and the authors provide an open-source release to support reproducibility.

Abstract

Visual SLAM in dynamic environments remains challenging, as several existing methods rely on semantic filtering that only handles known object classes, or use fixed robust kernels that cannot adapt to unknown moving objects, leading to degraded accuracy when they appear in the scene. We present VAR-SLAM (Visual Adaptive and Robust SLAM), an ORB-SLAM3-based system that combines a lightweight semantic keypoint filter to deal with known moving objects, with Barron's adaptive robust loss to handle unknown ones. The shape parameter of the robust kernel is estimated online from residuals, allowing the system to automatically adjust between Gaussian and heavy-tailed behavior. We evaluate VAR-SLAM on the TUM RGB-D, Bonn RGB-D Dynamic, and OpenLORIS datasets, which include both known and unknown moving objects. Results show improved trajectory accuracy and robustness over state-of-the-art baselines, achieving up to 25% lower ATE RMSE than NGD-SLAM on challenging sequences, while maintaining performance at 27 FPS on average.

Figures (9)

• Figure 1: Overview of our approach. Known dynamic objects (person) are filtered using semantic detection, while unknown dynamic features (balloon) are handled by adaptive robust kernels. Together, these modules enable robust SLAM in dynamic environments.
• Figure 2: System framework. Our method extends ORB-SLAM3 by introducing semantic filtering to remove features on known dynamic objects, and adaptive robust kernels for handling unknown dynamics. Pose optimization uses a fixed $\alpha$=1 for stable tracking and efficiency, while Local BA employs alternating optimization with the Barron kernel. Global BA is left unchanged.
• Figure 3: Dynamic keypoint filtering. (a) ORB keypoints before filtering. (b) keypoints filtered inside the bounding box from YOLOv4 detection. (c) keypoint removal with depth-awareness: keypoints whose depth is consistent with the foreground person are discarded.
• Figure 4: Feature extraction examples from the moving_obs_box sequence, where a large box occupies a large portion of the image.
• Figure 5: Output trajectories of (a) NGD-SLAM and (b) VAR-SLAM in the moving_obs_box sequence.