RadarScenes: A Real-World Radar Point Cloud Data Set for Automotive Applications

TL;DR

RadarScenes tackles the need for large-scale, real-world radar data for autonomous driving by introducing a comprehensive automotive radar point cloud dataset collected with four 77 GHz sensors. It provides 158 sequences, over four hours of driving, 11 object classes, manual per-detection and track annotations, and accompanying documentation camera imagery, all organized for easy processing in HDF5 and linked via a scenes map. An explicit evaluation protocol for object detection, classification, clustering, and semantic segmentation is proposed, with metrics such as mAP50, LAMR, and macro F1 to enable fair cross-method comparisons across accumulation schemes. By focusing on moving road users and offering detailed labeling and robust statistics, RadarScenes aims to catalyze radar-only perception and multi-sensor fusion research in automotive contexts.

Abstract

A new automotive radar data set with measurements and point-wise annotations from more than four hours of driving is presented. Data provided by four series radar sensors mounted on one test vehicle were recorded and the individual detections of dynamic objects were manually grouped to clusters and labeled afterwards. The purpose of this data set is to enable the development of novel (machine learning-based) radar perception algorithms with the focus on moving road users. Images of the recorded sequences were captured using a documentary camera. For the evaluation of future object detection and classification algorithms, proposals for score calculation are made so that researchers can evaluate their algorithms on a common basis. Additional information as well as download instructions can be found on the website of the data set: www.radar-scenes.com.

Figures (4)

• Figure 1: This article introduces the first diverse large-scale data set for automotive radar point clouds. It comprises over 4 hours and 100 of driving at a total of 158 different sequences with over 7500 manually annotated unique road users from 11 different object classes.
• Figure 2: Radar bird's-eye view point cloud images and corresponding camera images. Orientation and magnitude of each point's estimated radial velocity after ego-velocity compensation are depicted by gray arrows. In the documentation camera images, road users are masked due to EU regulations (GDPR). Images allow for intensive zooming.
• Figure 3: All sensor measurements are given in car coordinates ($cc$) with the origin located at the rear axle center. The documentary camera is mounted behind the windscreen (purple). The fields of view for each of the radar sensors are shown color-coded including their respective sensor ids.
• Figure 4: Data set statistics for the five main object classes: (\ref{['fig:class_dist']}) indicates the amount of annotated points and individual objects for each class. Fig. (\ref{['fig:frame_dist']}) shows how points and instances are distributed over the sensors scans. In (\ref{['fig:distance_dist']}), the normed distributions of all five classes are displayed for ranges up to 100m.