Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adver-City: Open-Source Multi-Modal Dataset for Collaborative Perception Under Adverse Weather Conditions

Mateus Karvat, Sidney Givigi

TL;DR

Adver-City tackles the scarcity of adverse-weather data for collaborative perception by providing the first open-source, multi-modal CP dataset generated in CARLA with OpenCDA. It offers 110 scenarios across six weather conditions, five road configurations, and two density levels, captured from five viewpoints with six object categories, enabling CP tasks under occlusion and degraded visibility. Benchmark results show models trained on Adver-City generalize better to adverse weather than those trained only on weather-free data, while still highlighting the persistent challenges of perception in harsh conditions. By analyzing realism via t-SNE and FID against real-world datasets, the work demonstrates that Adver-City is a viable research resource that can be extended and improved through community contributions and additional weather or map variations.

Abstract

Adverse weather conditions pose a significant challenge to the widespread adoption of Autonomous Vehicles (AVs) by impacting sensors like LiDARs and cameras. Even though Collaborative Perception (CP) improves AV perception in difficult conditions, existing CP datasets lack adverse weather conditions. To address this, we introduce Adver-City, the first open-source synthetic CP dataset focused on adverse weather conditions. Simulated in CARLA with OpenCDA, it contains over 24 thousand frames, over 890 thousand annotations, and 110 unique scenarios across six different weather conditions: clear weather, soft rain, heavy rain, fog, foggy heavy rain and, for the first time in a synthetic CP dataset, glare. It has six object categories including pedestrians and cyclists, and uses data from vehicles and roadside units featuring LiDARs, RGB and semantic segmentation cameras, GNSS, and IMUs. Its scenarios, based on real crash reports, depict the most relevant road configurations for adverse weather and poor visibility conditions, varying in object density, with both dense and sparse scenes, allowing for novel testing conditions of CP models. Benchmarks run on the dataset show that weather conditions created challenging conditions for perception models, with CoBEVT scoring 58.30/52.44/38.90 (AP@30/50/70). The dataset, code and documentation are available at https://labs.cs.queensu.ca/quarrg/datasets/adver-city/.

Adver-City: Open-Source Multi-Modal Dataset for Collaborative Perception Under Adverse Weather Conditions

TL;DR

Adver-City tackles the scarcity of adverse-weather data for collaborative perception by providing the first open-source, multi-modal CP dataset generated in CARLA with OpenCDA. It offers 110 scenarios across six weather conditions, five road configurations, and two density levels, captured from five viewpoints with six object categories, enabling CP tasks under occlusion and degraded visibility. Benchmark results show models trained on Adver-City generalize better to adverse weather than those trained only on weather-free data, while still highlighting the persistent challenges of perception in harsh conditions. By analyzing realism via t-SNE and FID against real-world datasets, the work demonstrates that Adver-City is a viable research resource that can be extended and improved through community contributions and additional weather or map variations.

Abstract

Adverse weather conditions pose a significant challenge to the widespread adoption of Autonomous Vehicles (AVs) by impacting sensors like LiDARs and cameras. Even though Collaborative Perception (CP) improves AV perception in difficult conditions, existing CP datasets lack adverse weather conditions. To address this, we introduce Adver-City, the first open-source synthetic CP dataset focused on adverse weather conditions. Simulated in CARLA with OpenCDA, it contains over 24 thousand frames, over 890 thousand annotations, and 110 unique scenarios across six different weather conditions: clear weather, soft rain, heavy rain, fog, foggy heavy rain and, for the first time in a synthetic CP dataset, glare. It has six object categories including pedestrians and cyclists, and uses data from vehicles and roadside units featuring LiDARs, RGB and semantic segmentation cameras, GNSS, and IMUs. Its scenarios, based on real crash reports, depict the most relevant road configurations for adverse weather and poor visibility conditions, varying in object density, with both dense and sparse scenes, allowing for novel testing conditions of CP models. Benchmarks run on the dataset show that weather conditions created challenging conditions for perception models, with CoBEVT scoring 58.30/52.44/38.90 (AP@30/50/70). The dataset, code and documentation are available at https://labs.cs.queensu.ca/quarrg/datasets/adver-city/.
Paper Structure (11 sections, 11 figures, 7 tables)

This paper contains 11 sections, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adver-City Dataset
  4. Scenarios
  5. Dataset Generation
  6. Dataset Statistics
  7. Benchmark
  8. Limitations
  9. Conclusions
  10. Road Configurations
  11. Comparison with Real-Life Datasets

Figures (11)

  • Figure 1: Camera samples from Adver-City showcasing its weather diversity. (a) Heavy rain night at an urban intersection. (b) Foggy heavy rain day at a rural straight non-junction. (c) Clear night at a rural intersection. (d) Soft rain day at a rural curved non-junction. (e) Glare day at an urban intersection. (f) Foggy night at an urban non-junction.
  • Figure 2: Camera samples from the different viewpoints used in the Adver-City dataset. Scenario: glare day at urban non-junction. Top: RSUs, center: ego AV, bottom: connected AVs. Despite the ego's front camera being heavily affected by glare, the RSUs remain unaffected, providing valuable information about the ego's surroundings.
  • Figure 3: Sensor positioning in CAVs. Camera positioning indicates both RGB and semantic segmentation cameras. Side-facing cameras are tilted backwards by $10^\circ$ to reduce motion blur effects.
  • Figure 4: Statistics for Adver-City. (a) Percentage of frames pertaining to each weather condition. (b) Percentage of frames per time of day. (c) Number of objects within line of sight of ego vehicle per distance to ego on dense and sparse scenarios. (d) Number of keyframes per annotation counts. (e) Polar density map in log scale for objects within line of sight of ego vehicle. Distance (in meters) and angle are in relation to ego, with the scale shown to the right. (f) Number of annotated instances per object class. Objects that are simultaneously detected by multiple viewpoints are not counted multiple times.
  • Figure 5: t-SNE plot generated with Scikit-Learn pedregosa_2011_scikit-learn from features computed by EfficientNet tan_2019_efficientnet of each dataset's rainy images. Adver-City's clusters have lower overlap than the real-world datasets, yet still having minor overlap with them.
  • ...and 6 more figures