Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Collective Perception Datasets for Autonomous Driving: A Comprehensive Review

Sven Teufel, Jörg Gamerdinger, Jan-Patrick Kirchner, Georg Volk, Oliver Bringmann

TL;DR

This paper addresses the need for comprehensive datasets to enable collective perception in autonomous driving, consolidating V2V/V2X and infrastructure-based data into simulated and real-world categories. It systematically catalogs existing datasets, compares their sensor modalities, scenario diversity, and annotation richness, and evaluates their suitability for 3D object detection, tracking, and semantic segmentation. The work highlights strengths, limitations, and anomalies across datasets, and provides guidance on selecting datasets to advance collective perception research while acknowledging sim-to-real gaps. By offering a structured taxonomy and critical comparisons, the paper informs researchers and practitioners about how to choose appropriate benchmarks and where to focus future data collection and standardization efforts for robust connected automated vehicle systems.

Abstract

To ensure safe operation of autonomous vehicles in complex urban environments, complete perception of the environment is necessary. However, due to environmental conditions, sensor limitations, and occlusions, this is not always possible from a single point of view. To address this issue, collective perception is an effective method. Realistic and large-scale datasets are essential for training and evaluating collective perception methods. This paper provides the first comprehensive technical review of collective perception datasets in the context of autonomous driving. The survey analyzes existing V2V and V2X datasets, categorizing them based on different criteria such as sensor modalities, environmental conditions, and scenario variety. The focus is on their applicability for the development of connected automated vehicles. This study aims to identify the key criteria of all datasets and to present their strengths, weaknesses, and anomalies. Finally, this survey concludes by making recommendations regarding which dataset is most suitable for collective 3D object detection, tracking, and semantic segmentation.

Collective Perception Datasets for Autonomous Driving: A Comprehensive Review

TL;DR

This paper addresses the need for comprehensive datasets to enable collective perception in autonomous driving, consolidating V2V/V2X and infrastructure-based data into simulated and real-world categories. It systematically catalogs existing datasets, compares their sensor modalities, scenario diversity, and annotation richness, and evaluates their suitability for 3D object detection, tracking, and semantic segmentation. The work highlights strengths, limitations, and anomalies across datasets, and provides guidance on selecting datasets to advance collective perception research while acknowledging sim-to-real gaps. By offering a structured taxonomy and critical comparisons, the paper informs researchers and practitioners about how to choose appropriate benchmarks and where to focus future data collection and standardization efforts for robust connected automated vehicle systems.

Abstract

To ensure safe operation of autonomous vehicles in complex urban environments, complete perception of the environment is necessary. However, due to environmental conditions, sensor limitations, and occlusions, this is not always possible from a single point of view. To address this issue, collective perception is an effective method. Realistic and large-scale datasets are essential for training and evaluating collective perception methods. This paper provides the first comprehensive technical review of collective perception datasets in the context of autonomous driving. The survey analyzes existing V2V and V2X datasets, categorizing them based on different criteria such as sensor modalities, environmental conditions, and scenario variety. The focus is on their applicability for the development of connected automated vehicles. This study aims to identify the key criteria of all datasets and to present their strengths, weaknesses, and anomalies. Finally, this survey concludes by making recommendations regarding which dataset is most suitable for collective 3D object detection, tracking, and semantic segmentation.
Paper Structure (34 sections, 3 figures, 1 table)

This paper contains 34 sections, 3 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. DATASET REVIEW
  3. Simulated Datasets
  4. V2V-Sim
  5. CODD
  6. COMAP
  7. V2XSet
  8. DOLPHINS
  9. OPV2V
  10. V2X-Sim
  11. IRV2V
  12. DeepAccident
  13. Coop3DInf
  14. Real-World Datasets:
  15. T&J
  16. ...and 19 more sections

Figures (3)

  • Figure 1: Collective LiDAR point cloud from CODD codd
  • Figure 2: Exemplary scenes from simulated datasets with images. DOLPHINS (top left), OPV2V (top right), V2XSet (bottom left), and V2X-Sim (bottom right)
  • Figure 3: Overview of the presented V2V and V2X datasets. Datasets where information are missing and infrastructure-only datasets are not shown here, an overview on all V2V and V2X datasets is shown in Tab. \ref{['tab:overview']}.