RACECAR -- The Dataset for High-Speed Autonomous Racing

TL;DR

The paper introduces RACECAR, a pioneering open dataset of multi-modal sensor data from fully autonomous Indy race cars at speeds up to $170$ mph, collected across 11 racing scenarios on two tracks and totaling over 6.5 hours. It provides data in both ROS2 bag and nuScenes formats, along with a ROS2-to-nuScenes conversion tool, enabling broad accessibility and benchmarking. Key contributions include centimeter-accurate ground-truth pose, 27 racing sessions, and three baseline benchmarks for localization, object detection/tracking, and mapping across LiDAR, Radar, and Camera modalities. By democratizing access to high-speed autonomous racing data and offering processing tools, RACECAR aims to advance perception, planning, and control algorithms that operate robustly at the limits of vehicle capability.

Abstract

This paper describes the first open dataset for full-scale and high-speed autonomous racing. Multi-modal sensor data has been collected from fully autonomous Indy race cars operating at speeds of up to 170 mph (273 kph). Six teams who raced in the Indy Autonomous Challenge have contributed to this dataset. The dataset spans 11 interesting racing scenarios across two race tracks which include solo laps, multi-agent laps, overtaking situations, high-accelerations, banked tracks, obstacle avoidance, pit entry and exit at different speeds. The dataset contains data from 27 racing sessions across the 11 scenarios with over 6.5 hours of sensor data recorded from the track. The data is organized and released in both ROS2 and nuScenes format. We have also developed the ROS2-to-nuScenes conversion library to achieve this. The RACECAR data is unique because of the high-speed environment of autonomous racing. We present several benchmark problems on localization, object detection and tracking (LiDAR, Radar, and Camera), and mapping using the RACECAR data to explore issues that arise at the limits of operation of the vehicle.

Figures (10)

• Figure 1: RACECAR is the first multi-model sensor data collected from fully autonomous Indy race cars operating at speeds of up to 170 mph (274 kph). The dataset spans 11 racing scenarios with over 6.5 hours of track activity.
• Figure 2: [Left] The AV-21 is a modified Indy Lights racecar retrofitted with 3 LiDARs, 6 Cameras, 3 Radars, 2 GNSS systems. [Right] The Indy Autonomous Challenge (IAC) held its first autonomous race at the Indianapolis Motor Speedway (IMS) track in 2021, followed by a head-to-head overtaking competition held at the Las Vegas Motor Speedway (LVMS).
• Figure 3: ROS2 bags to nuScenes conversion process
• Figure 4: [Left] Localization Pipeline, sensors produce state measurements which are used as updates for an Extended Kalman Filter to produce a vehicle Pose estimate. [Right] The blue dots represent GNSS updates at 20 Hz. The red dots are EKF predictions at 100 Hz. At approximately 40 m/s, the car travels 2 meters before another GNSS update.
• Figure 5: LiDAR Perception: Processing raw LiDAR point clouds using Range Based Filtering, Ground Plane Detection, Wall Detection, and Euclidean Based Clustering