Design and Evaluation of a Generic Visual SLAM Framework for Multi-Camera Systems

TL;DR

This work addresses the challenge of robust and scalable SLAM for mobile robots equipped with arbitrary multi-camera rigs by introducing a real-time generic visual SLAM framework that models the rig as a generalized camera and exploits cross-camera overlaps to extract multi-view features. The front-end fuses overlapping views into cross-camera features while the back-end performs MAP optimization over poses, landmarks, and camera extrinsics using a factor-graph formalism. Key contributions include a complete, configuration-independent SLAM pipeline, a systematic evaluation of camera count and FoV overlap, and publicly available datasets and code. Results show that increasing FoV overlap and the number of cameras generally improves accuracy and robustness in OV configurations, with N-OV setups suffering scale drift; the framework demonstrates near real-time performance on practical rigs. This work advances practical deployment of multi-camera SLAM by providing design insights for camera configuration and a scalable, extensible software stack.

Abstract

Multi-camera systems have been shown to improve the accuracy and robustness of SLAM estimates, yet state-of-the-art SLAM systems predominantly support monocular or stereo setups. This paper presents a generic sparse visual SLAM framework capable of running on any number of cameras and in any arrangement. Our SLAM system uses the generalized camera model, which allows us to represent an arbitrary multi-camera system as a single imaging device. Additionally, it takes advantage of the overlapping fields of view (FoV) by extracting cross-matched features across cameras in the rig. This limits the linear rise in the number of features with the number of cameras and keeps the computational load in check while enabling an accurate representation of the scene. We evaluate our method in terms of accuracy, robustness, and run time on indoor and outdoor datasets that include challenging real-world scenarios such as narrow corridors, featureless spaces, and dynamic objects. We show that our system can adapt to different camera configurations and allows real-time execution for typical robotic applications. Finally, we benchmark the impact of the critical design parameters - the number of cameras and the overlap between their FoV that define the camera configuration for SLAM. All our software and datasets are freely available for further research.

Figures (7)

• Figure 1: (a) Illustration of various overlapping(OV) and non-overlapping(N-OV) camera configurations evaluated in this work. (b) Block diagram of the generic visual SLAM framework showing its sub-modules. Feature extraction computes two types of features– multi-view intra-matches and regular mono features. Note the changes, made to initialization, tracking and keyframe representation to adapt to a general multi-camera system.
• Figure 2: Two sample multi-camera frames with extracted features on (a) front-facing cameras in overlapping (OV) and, (b) non-overlapping (N-OV) setups for the same scene. The multi-view metric features are colored based on their distance to the camera system. The white points are mono features that do not have any 3D information. Notice that N-OV setup has only mono features, whereas the OV setup has both multi-view and mono features distributed across OV and N-OV regions in the images.
• Figure 3: Factor graph of the multi-camera back-end with poses $X_i$, landmarks $l_j$ and the relative camera poses $C_p$ as the variables to be optimized. The black square factor nodes represent constraints on the variables.
• Figure 4: The custom-built multi-camera rig used to collect data for evaluating the SLAM pipeline. The figure shows overlapping and non-overlapping configurations and the IMU that was mounted on the rig. The IMU is used to compute the baseline between two consecutive cameras is 165mm.
• Figure 5: Estimated trajectories of the Curry_center sequence with outdoor data and dynamic content. Stars indicate final positions of trajectory estimates. Accuracy and robustness improve with increasing number of cameras in OV configurations, as shown by accumulated drift in final position. Red and blue boxes highlight tracking failures caused by occluding dynamic objects. N-OV configuration exhibits scale issues compared to OV configuration but is robust to dynamic content.