Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SLAM Adversarial Lab: An Extensible Framework for Visual SLAM Robustness Evaluation under Adverse Conditions

Mohamed Hefny, Karthik Dantu, Steven Y. Ko

Abstract

We present SAL (SLAM Adversarial Lab), a modular framework for evaluating visual SLAM systems under adversarial conditions such as fog and rain. SAL represents each adversarial condition as a perturbation that transforms an existing dataset into an adversarial dataset. When transforming a dataset, SAL supports severity levels using easily-interpretable real-world units such as meters for fog visibility. SAL's extensible architecture decouples datasets, perturbations, and SLAM algorithms through common interfaces, so users can add new components without rewriting integration code. Moreover, SAL includes a search procedure that finds the severity level of a perturbation at which a SLAM system fails. To showcase the capabilities of SAL, our evaluation integrates seven SLAM algorithms and evaluates them across three datasets under weather, camera, and video transport perturbations.

SLAM Adversarial Lab: An Extensible Framework for Visual SLAM Robustness Evaluation under Adverse Conditions

Abstract

We present SAL (SLAM Adversarial Lab), a modular framework for evaluating visual SLAM systems under adversarial conditions such as fog and rain. SAL represents each adversarial condition as a perturbation that transforms an existing dataset into an adversarial dataset. When transforming a dataset, SAL supports severity levels using easily-interpretable real-world units such as meters for fog visibility. SAL's extensible architecture decouples datasets, perturbations, and SLAM algorithms through common interfaces, so users can add new components without rewriting integration code. Moreover, SAL includes a search procedure that finds the severity level of a perturbation at which a SLAM system fails. To showcase the capabilities of SAL, our evaluation integrates seven SLAM algorithms and evaluates them across three datasets under weather, camera, and video transport perturbations.
Paper Structure (24 sections, 4 figures, 4 tables)

This paper contains 24 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. SLAM Evaluation Frameworks
  4. Adverse Condition Datasets
  5. Robust SLAM Algorithms
  6. SAL (SLAM Adversarial Lab)
  7. Perturbation Modules
  8. Extensible Benchmarking Architecture
  9. Robustness Boundary Evaluation
  10. Search Procedure
  11. Extensibility
  12. Experiment Execution
  13. Perturbation Effects
  14. Weather and Illumination Conditions
  15. Camera Conditions
  16. ...and 9 more sections

Figures (4)

  • Figure 1: System overview and data flow. Experiment configuration and dataset adapters feed the perturbation pipeline, which outputs perturbed sequences. The original and perturbed sequences are evaluated by the SLAM pipeline (trajectories, metrics, plots) and the odometry pipeline (tracking statistics). The robustness-boundary pipeline reuses perturbation and SLAM pipelines to localize the failure boundary.
  • Figure 2: Example perturbations applied to KITTI sequences. Table \ref{['tab:exp_params']} lists representative parameter values for the severity levels.
  • Figure 3: Outdoor (KITTI) SLAM trajectory evaluation in the X-Z plane. HTML]2ca02c baseline, HTML]6baed6 Light, HTML]fed976 Moderate, HTML]fd8d3c Heavy, HTML]d73027 Severe.
  • Figure 4: Indoor (TUM left, EuRoC right) SLAM trajectory evaluation in the X-Y plane. Legend as in Fig. \ref{['fig:slam_results_kitti']}.