Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SNAIL Radar: A large-scale diverse benchmark for evaluating 4D-radar-based SLAM

Jianzhu Huai, Binliang Wang, Yuan Zhuang, Yiwen Chen, Qipeng Li, Yulong Han

TL;DR

The snail-radar dataset addresses the need for large-scale, multi-platform 4D radar SLAM benchmarks by collecting 44 sequences across 8 routes on handheld, e-bike, and SUV platforms under diverse weather and lighting. It provides TLS-based, forward/backward reference trajectories and a cascaded pose graph optimization framework to produce accurate full trajectories, with a robust synchronization pipeline aligning all sensors to GNSS time. The work also delivers rich baselines by evaluating recent radar odometry and place recognition methods, revealing current methods’ drift and robustness gaps, and highlighting the dataset’s value for SLAM and place recognition research as well as potential neural 3D reconstruction benchmarking. This dataset thus offers a practical, open resource for rigorous radar-based localization and mapping evaluation in challenging real-world conditions, enabling researchers to quantify performance, reproduce results, and drive improvements.

Abstract

4D radars are increasingly favored for odometry and mapping of autonomous systems due to their robustness in harsh weather and dynamic environments. Existing datasets, however, often cover limited areas and are typically captured using a single platform. To address this gap, we present a diverse large-scale dataset specifically designed for 4D radar-based localization and mapping. This dataset was gathered using three different platforms: a handheld device, an e-bike, and an SUV, under a variety of environmental conditions, including clear days, nighttime, and heavy rain. The data collection occurred from September 2023 to February 2024, encompassing diverse settings such as roads in a vegetated campus and tunnels on highways. Each route was traversed multiple times to facilitate place recognition evaluations. The sensor suite included a 3D lidar, 4D radars, stereo cameras, consumer-grade IMUs, and a GNSS/INS system. Sensor data packets were synchronized to GNSS time using a two-step process including a convex-hull-based smoothing and a correlation-based correction. The reference motion for the platforms was generated by registering lidar scans to a terrestrial laser scanner (TLS) point cloud map by a lidar inertial sequential localizer which supports forward and backward processing. The backward pass enables detailed quantitative and qualitative assessments of reference motion accuracy. To demonstrate the dataset's utility, we evaluated several state-of-the-art radar-based odometry and place recognition methods, indicating existing challenges in radar-based SLAM.

SNAIL Radar: A large-scale diverse benchmark for evaluating 4D-radar-based SLAM

TL;DR

The snail-radar dataset addresses the need for large-scale, multi-platform 4D radar SLAM benchmarks by collecting 44 sequences across 8 routes on handheld, e-bike, and SUV platforms under diverse weather and lighting. It provides TLS-based, forward/backward reference trajectories and a cascaded pose graph optimization framework to produce accurate full trajectories, with a robust synchronization pipeline aligning all sensors to GNSS time. The work also delivers rich baselines by evaluating recent radar odometry and place recognition methods, revealing current methods’ drift and robustness gaps, and highlighting the dataset’s value for SLAM and place recognition research as well as potential neural 3D reconstruction benchmarking. This dataset thus offers a practical, open resource for rigorous radar-based localization and mapping evaluation in challenging real-world conditions, enabling researchers to quantify performance, reproduce results, and drive improvements.

Abstract

4D radars are increasingly favored for odometry and mapping of autonomous systems due to their robustness in harsh weather and dynamic environments. Existing datasets, however, often cover limited areas and are typically captured using a single platform. To address this gap, we present a diverse large-scale dataset specifically designed for 4D radar-based localization and mapping. This dataset was gathered using three different platforms: a handheld device, an e-bike, and an SUV, under a variety of environmental conditions, including clear days, nighttime, and heavy rain. The data collection occurred from September 2023 to February 2024, encompassing diverse settings such as roads in a vegetated campus and tunnels on highways. Each route was traversed multiple times to facilitate place recognition evaluations. The sensor suite included a 3D lidar, 4D radars, stereo cameras, consumer-grade IMUs, and a GNSS/INS system. Sensor data packets were synchronized to GNSS time using a two-step process including a convex-hull-based smoothing and a correlation-based correction. The reference motion for the platforms was generated by registering lidar scans to a terrestrial laser scanner (TLS) point cloud map by a lidar inertial sequential localizer which supports forward and backward processing. The backward pass enables detailed quantitative and qualitative assessments of reference motion accuracy. To demonstrate the dataset's utility, we evaluated several state-of-the-art radar-based odometry and place recognition methods, indicating existing challenges in radar-based SLAM.
Paper Structure (16 sections, 1 equation, 4 figures, 8 tables, 1 algorithm)

This paper contains 16 sections, 1 equation, 4 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Radar signal processing background
  4. Dataset
  5. Sensor setups
  6. File formats
  7. Reference trajectories
  8. TLS-based lidar inertial localization
  9. Cascaded pose graph optimization
  10. Synchronization
  11. Sensor rig calibration
  12. Known issues
  13. Comparative studies
  14. Radar odometry
  15. Radar place recognition
  16. ...and 1 more sections

Figures (4)

  • Figure 1: (Top) The schematic of the millimeter wave radar's mechanism, as in a typical 4D radar by Texas Instruments (TI) iovescuFundamentalsMillimeterWave2017. PA: power amplifier, BPM: binary phase modulation, LNA: low noise amplifier, IF: intermediate frequency, ADC: analog-to-digital converter, DSP: digital signal processor, AMP: amplifier. (Bottom) The simplified radar point cloud generation pipeline from the ADC samples. CFAR: constant false alarm rate. The math symbols are explained in the main text.
  • Figure 2: The routes used for data collection in our dataset. The top plot shows four routes: the basketball court (yellow), the starlake (pink), the software school (brown), and the starlake tower (blue). The bottom plot shows the other four routes: the info arts and engineering faculty (pink), the info and arts faculty (yellow), the August 1 road (blue), and the info faculty (brown). We slightly offset the paths for clarity.
  • Figure 3: The sensor rig on three platforms and the coordinate frames of the sensors. The Thinkpad P53 laptop and the bundle of cables are concealed in a waterproof bag.
  • Figure 4: The translation (a, c) and rotation (b, d) differences between the forward (or backward) localization and their averages are shown for the e-bike sequences 20231105/4 (a, b) and 20231105_aft/4 (c, d). These two sequences exhibit the largest maximum translation deviations among all sequences in the dataset.