ACDC: The Adverse Conditions Dataset with Correspondences for Robust Semantic Driving Scene Perception

TL;DR

ACDC addresses the lack of large-scale, high-quality pixel-level data for driving scene perception in adverse conditions by providing 5509 panoptic-annotated images (4006 adverse across fog, night, rain, snow; 1503 normal references) plus privileged correspondences. It supports semantic segmentation, instance segmentation, panoptic segmentation, object detection, and uncertainty-aware semantic segmentation via a two-stage annotation protocol that marks invalid regions and a dedicated AUIoU metric. The paper demonstrates the dataset’s value across supervised and unsupervised learning, normal-to-adverse and sensor-level domain adaptation, and externally pre-trained model evaluation, showing that adversarial domains remain challenging and that privileged information and cross-condition correspondences can meaningfully aid learning and evaluation. Overall, ACDC serves as a realistic, scalable benchmark that drives progress in robust driving perception under adverse visual conditions and encourages uncertainty-aware, domain-adaptive approaches with real-world impact.

Abstract

Level-5 driving automation requires a robust visual perception system that can parse input images under any condition. However, existing driving datasets for dense semantic perception are either dominated by images captured under normal conditions or are small in scale. To address this, we introduce ACDC, the Adverse Conditions Dataset with Correspondences for training and testing methods for diverse semantic perception tasks on adverse visual conditions. ACDC consists of a large set of 8012 images, half of which (4006) are equally distributed between four common adverse conditions: fog, nighttime, rain, and snow. Each adverse-condition image comes with a high-quality pixel-level panoptic annotation, a corresponding image of the same scene under normal conditions, and a binary mask that distinguishes between intra-image regions of clear and uncertain semantic content. 1503 of the corresponding normal-condition images feature panoptic annotations, raising the total annotated images to 5509. ACDC supports the standard tasks of semantic segmentation, object detection, instance segmentation, and panoptic segmentation, as well as the newly introduced uncertainty-aware semantic segmentation. A detailed empirical study demonstrates the challenges that the adverse domains of ACDC pose to state-of-the-art supervised and unsupervised approaches and indicates the value of our dataset in steering future progress in the field. Our dataset and benchmark are publicly available at https://acdc.vision.ee.ethz.ch

Figures (7)

• Figure 1: Number of finely annotated pixels per class in ACDC.
• Figure 2: Illustration of construction and semantic annotation pipeline for ACDC. The color coding of the semantic classes matches Fig. \ref{['fig:dataset:stats']}. All three annotation outputs pertain to the input adverse-condition image $I$. A white color in the draft semantic annotation from stage 1 and the final semantic annotation $H$ from stage 2 denotes unlabeled pixels. Dashed lines denote privileged information beyond the input adverse-condition image $I$ under annotation; this information is additionally leveraged in annotation stage 2.
• Figure 3: Number of instances per class in ACDC.
• Figure 4: Qualitative results of selected semantic segmentation methods on ACDC. From left to right: image, ground-truth annotation, FDA fda:adaptation, DeepLabv3+ DeepLab:v3+, and HRNet hrnet. The color coding of the semantic classes matches Fig. \ref{['fig:dataset:stats']}.
• Figure 5: Qualitative results of panoptic segmentation methods on ACDC. From left to right: image, Panoptic FPN Kirillov_2019_CVPR_panoptic_fpn, K-Net NEURIPS2021_knet, Panoptic-Deeplab cheng2020panoptic_deeplab, and Mask2Former cheng2021mask2former. The color coding of the semantic classes matches Fig. \ref{['fig:dataset:stats']}.