Open-Structure: Structural Benchmark Dataset for SLAM Algorithms

TL;DR

A baseline is proposed using the Open-Structure dataset to evaluate widely used modules, including camera pose tracking, parametrization, and factor graph optimization, within SLAM systems, to discern each module's strengths and weaknesses in the context of camera tracking and optimization processes.

Abstract

This paper presents Open-Structure, a novel benchmark dataset for evaluating visual odometry and SLAM methods. Compared to existing public datasets that primarily offer raw images, Open-Structure provides direct access to point and line measurements, correspondences, structural associations, and co-visibility factor graphs, which can be fed to various stages of SLAM pipelines to mitigate the impact of data preprocessing modules in ablation experiments. The dataset comprises two distinct types of sequences from the perspective of scenarios. The first type maintains reasonable observation and occlusion relationships, as these critical elements are extracted from public image-based sequences using our dataset generator. In contrast, the second type consists of carefully designed simulation sequences that enhance dataset diversity by introducing a wide range of trajectories and observations. Furthermore, a baseline is proposed using our dataset to evaluate widely used modules, including camera pose tracking, parametrization, and factor graph optimization, within SLAM systems. By evaluating these state-of-the-art algorithms across different scenarios, we discern each module's strengths and weaknesses in the context of camera tracking and optimization processes. The Open-Structure dataset and baseline system are openly accessible on website: \url{https://open-structure.github.io}, encouraging further research and development in the field of SLAM.

Figures (10)

• Figure 1: Examples of data provided in Open-Structure Benchmark Dataset.
• Figure 2: Statistical analysis per frame of the sphere2 sequence.
• Figure 3: Comparison in positions of detected features (blue) and generated measurements (green) of our dataset.
• Figure 4: Overview of trajectories, 3D models and 2D measurements of Open-Structure sequences. Point and line measurements are colored in green and red, respectively. (a) office0 comes from S-I showing reasonable feature distributions. (b) box1 comes from S-II presenting more challenging trajectories and observations.
• Figure 5: The architecture of the Open-Structure baseline that reads point and line measurements directly. Initial camera poses, and a sparse map are estimated simultaneously via 3D-2D alignment and landmark fusion blocks. Co-visibility observations, initial poses, and landmarks are fed to the Co-visibility Graph Optimization module, where initial, optimized, and ground truth landmarks are highlighted in dark green, light green, and red, respectively.