CARLA-Loc: Synthetic SLAM Dataset with Full-stack Sensor Setup in Challenging Weather and Dynamic Environments

TL;DR

CARLA-Loc addresses the need for controlled evaluation of SLAM under dynamic objects and diverse weather by providing a synthetic, full-stack SLAM dataset built in the CARLA simulator. It features 7 maps and 42 sequences with tuned multi-sensor setups and a pipeline to generate custom sequences and export ROS bags, ensuring identical ego motion across conditions. The paper benchmarks 5 visual-based and 4 LiDAR-based SLAM methods, showing that IMU integration improves robustness while dynamic objects and adverse weather still challenge localization and mapping. This dataset and toolkit offer a practical, repeatable benchmark for developing and validating robust SLAM solutions in autonomous driving.

Abstract

The robustness of SLAM (Simultaneous Localization and Mapping) algorithms under challenging environmental conditions is critical for the success of autonomous driving. However, the real-world impact of such conditions remains largely unexplored due to the difficulty of altering environmental parameters in a controlled manner. To address this, we introduce CARLA-Loc, a synthetic dataset designed for challenging and dynamic environments, created using the CARLA simulator. Our dataset integrates a variety of sensors, including cameras, event cameras, LiDAR, radar, and IMU, etc. with tuned parameters and modifications to ensure the realism of the generated data. CARLA-Loc comprises 7 maps and 42 sequences, each varying in dynamics and weather conditions. Additionally, a pipeline script is provided that allows users to generate custom sequences conveniently. We evaluated 5 visual-based and 4 LiDAR-based SLAM algorithms across different sequences, analyzing how various challenging environmental factors influence localization accuracy. Our findings demonstrate the utility of the CARLA-Loc dataset in validating the efficacy of SLAM algorithms under diverse conditions.

Figures (5)

• Figure 1: Overview of CARLA-Loc dataset. (a)-(c) show 3 preset weathers (clear noon, foggy noon, rainy night) for each map. (d)-(f) are images captured from depth camera, segmentation and event camera, respectively. (g) is the image from the static environment, all data are collected in the static setup as well. (h) is one scan of LiDAR in the dynamic environment, while (i) is the point cloud from static setup.
• Figure 2: The pipeline to generate CARLA-Loc dataset. The ego motion is controlled through steering wheel and pedal manually, while the motion of other vehicles and pedestrians are automatically controlled by Traffic Manager of CARLA. The object motions and traffic light states are recorded by CARLA Recorder. The recordings are replayed in the given conditions to generate the diverse final dataset.
• Figure 3: Sensor layout and coordinate systems of CARLA-Loc.
• Figure 4: Trajectories of ORB-SLAM3 (stereo mode) in sequences from map 03 and map 07, the presence of dynamic object, as well as the poor weather condition evidently affect the overall localization performance.
• Figure 5: Relative Position Error of A-LOAM in map 02, map 05 and map 07. In the presence of dynamic objects (orange line), the estimation error of relative positions is evidently greater. However, this impact can be mitigated using semantic segmentation label to exclude dynamic points (green line).