Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UDTIRI: An Online Open-Source Intelligent Road Inspection Benchmark Suite

Sicen Guo, Jiahang Li, Yi Feng, Dacheng Zhou, Denghuang Zhang, Chen Chen, Shuai Su, Xingyi Zhu, Qijun Chen, Rui Fan

TL;DR

Addressing the lack of large-scale, well-annotated data for intelligent road inspection in urban digital twins, the paper introduces UDTIRI, an online benchmark and the first road pothole detection competition. It provides a dataset of 1,000 RGB images with pixel- and instance-level ground-truth in VOC/COCO formats and a test-submission workflow for evaluating object detection, semantic segmentation, and instance segmentation models. Extensive baselines are reported across 14 detectors, 30 segmenters, and 10 instance-seg models, highlighting Transformer-based methods' advantages for fine-grained pothole tasks and trade-offs between accuracy and speed. The benchmark is proposed as a catalyst to advance UDT techniques in IRI and to support ongoing and future competitions.

Abstract

In the nascent domain of urban digital twins (UDT), the prospects for leveraging cutting-edge deep learning techniques are vast and compelling. Particularly within the specialized area of intelligent road inspection (IRI), a noticeable gap exists, underscored by the current dearth of dedicated research efforts and the lack of large-scale well-annotated datasets. To foster advancements in this burgeoning field, we have launched an online open-source benchmark suite, referred to as UDTIRI. Along with this article, we introduce the road pothole detection task, the first online competition published within this benchmark suite. This task provides a well-annotated dataset, comprising 1,000 RGB images and their pixel/instance-level ground-truth annotations, captured in diverse real-world scenarios under different illumination and weather conditions. Our benchmark provides a systematic and thorough evaluation of state-of-the-art object detection, semantic segmentation, and instance segmentation networks, developed based on either convolutional neural networks or Transformers. We anticipate that our benchmark will serve as a catalyst for the integration of advanced UDT techniques into IRI. By providing algorithms with a more comprehensive understanding of diverse road conditions, we seek to unlock their untapped potential and foster innovation in this critical domain.

UDTIRI: An Online Open-Source Intelligent Road Inspection Benchmark Suite

TL;DR

Addressing the lack of large-scale, well-annotated data for intelligent road inspection in urban digital twins, the paper introduces UDTIRI, an online benchmark and the first road pothole detection competition. It provides a dataset of 1,000 RGB images with pixel- and instance-level ground-truth in VOC/COCO formats and a test-submission workflow for evaluating object detection, semantic segmentation, and instance segmentation models. Extensive baselines are reported across 14 detectors, 30 segmenters, and 10 instance-seg models, highlighting Transformer-based methods' advantages for fine-grained pothole tasks and trade-offs between accuracy and speed. The benchmark is proposed as a catalyst to advance UDT techniques in IRI and to support ongoing and future competitions.

Abstract

In the nascent domain of urban digital twins (UDT), the prospects for leveraging cutting-edge deep learning techniques are vast and compelling. Particularly within the specialized area of intelligent road inspection (IRI), a noticeable gap exists, underscored by the current dearth of dedicated research efforts and the lack of large-scale well-annotated datasets. To foster advancements in this burgeoning field, we have launched an online open-source benchmark suite, referred to as UDTIRI. Along with this article, we introduce the road pothole detection task, the first online competition published within this benchmark suite. This task provides a well-annotated dataset, comprising 1,000 RGB images and their pixel/instance-level ground-truth annotations, captured in diverse real-world scenarios under different illumination and weather conditions. Our benchmark provides a systematic and thorough evaluation of state-of-the-art object detection, semantic segmentation, and instance segmentation networks, developed based on either convolutional neural networks or Transformers. We anticipate that our benchmark will serve as a catalyst for the integration of advanced UDT techniques into IRI. By providing algorithms with a more comprehensive understanding of diverse road conditions, we seek to unlock their untapped potential and foster innovation in this critical domain.
Paper Structure (14 sections, 6 figures, 3 tables)

This paper contains 14 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Objection Detection Networks
  4. Semantic Segmentation Networks
  5. Instance Segmentation Networks
  6. UDTIRI Benchmark Suite
  7. Experimental Results
  8. Experimental Setup
  9. Evaluation Metrics
  10. Object Detection Network Performance
  11. Semantic Segmentation Network Performance
  12. Instance Segmentation Network Performance
  13. Discussion
  14. Conclusion

Figures (6)

  • Figure 1: Publication and citation trends for road pothole detection over the past decade. Conference papers are sourced from the Engineering Village database (webpage: engineeringvillage.com/), while journal articles and citations are sourced from the Web of Science database (webpage: webofscience.com).
  • Figure 2: Examples of the ground-truth annotations for the road pothole detection competition within the UDTIRI benchmark suite: (a) object detection; (b) semantic segmentation; (c) instance segmentation.
  • Figure 3: Dataset characteristics: (a) a histogram showing the distribution of pothole numbers; (b) a histogram showing the distribution of pothole scales.
  • Figure 4: Qualitative experimental results of object detection. The green areas in the image represent true-positive predictions, the blue areas represent false-positive predictions, and the red areas represent false-negative predictions.
  • Figure 5: Qualitative experimental results of semantic segmentation. The green areas in the image represent true-positive predictions, the blue areas represent false-positive predictions, and the red areas represent false-negative predictions.
  • ...and 1 more figures