RSUD20K: A Dataset for Road Scene Understanding In Autonomous Driving

TL;DR

RSUD20K tackles cross-geography generalization in road scene understanding by introducing a Bangladesh-focused dataset with 20,334 high-resolution images and 130K bounding boxes across 13 classes. The authors implement a three-stage data-engineering pipeline (manual, semi-automatic, fully automatic) to assemble the dataset and benchmark both standard detectors and large vision models as open-set annotators. Key findings show state-of-the-art detectors like YOLOv6 achieve the best performance (up to 73.7 mAP), while LVM-based labeling underperforms compared to ground-truth, underscoring the importance of data-centric labeling for geography-specific scenarios. RSUD20K reveals significant class imbalance and densely cluttered scenes with varied illumination, providing a challenging benchmark to advance perception systems for autonomous driving in diverse environments.

Abstract

Road scene understanding is crucial in autonomous driving, enabling machines to perceive the visual environment. However, recent object detectors tailored for learning on datasets collected from certain geographical locations struggle to generalize across different locations. In this paper, we present RSUD20K, a new dataset for road scene understanding, comprised of over 20K high-resolution images from the driving perspective on Bangladesh roads, and includes 130K bounding box annotations for 13 objects. This challenging dataset encompasses diverse road scenes, narrow streets and highways, featuring objects from different viewpoints and scenes from crowded environments with densely cluttered objects and various weather conditions. Our work significantly improves upon previous efforts, providing detailed annotations and increased object complexity. We thoroughly examine the dataset, benchmarking various state-of-the-art object detectors and exploring large vision models as image annotators.

Figures (9)

• Figure 1: RSUD20K for road scene understanding. The dataset consists of a total of 20334 images with 130K bounding box annotations of 13 different objects. Images are captured from the driving perspective of diverse road scenes, objects from different viewpoints, occlusions, as well as under various weather conditions.
• Figure 2: Overview of our data engine used to create RSUD20K. We use a three-stage approach, where $f_{\boldsymbol{\theta}}$ is YOLOv6-M6 detector trained to predict bounding boxes and class labels of objects.
• Figure 3: Dataset statistics of RSUD20K. Image level annotations statistics (top) and bounding box instances (bottom).
• Figure 4: Distribution of box instances in RSUD20K. Most images have a total of roughly 5 to 8 bounding boxes, demonstrating that there are many objects in the driving scenes.
• Figure 5: Visualization comparison of predictions by state-of-the-art object detectors on RSUD20K. First two rows show occlusions and viewpoint differences. Second two rows show cases of various weather conditions. Zoom-in for better details.