VISION Datasets: A Benchmark for Vision-based InduStrial InspectiON

TL;DR

This paper introduces VISION Datasets, a comprehensive benchmark for vision-based industrial inspection designed to address data availability, quality, and production complexity. It provides 14 diverse datasets with instance-segmentation labels across train/val/test splits, totaling 18k images and 44 defect types, sourced from Roboflow. Two competitions are organized—one focused on data-efficient defect detection and the other on data-generation for defect detection—aimed at advancing both model-centric and data-centric approaches. The work also covers curation, annotation, and leakage-aware splitting, and discusses insights from winners, potential research directions, and limitations.

Abstract

Despite progress in vision-based inspection algorithms, real-world industrial challenges -- specifically in data availability, quality, and complex production requirements -- often remain under-addressed. We introduce the VISION Datasets, a diverse collection of 14 industrial inspection datasets, uniquely poised to meet these challenges. Unlike previous datasets, VISION brings versatility to defect detection, offering annotation masks across all splits and catering to various detection methodologies. Our datasets also feature instance-segmentation annotation, enabling precise defect identification. With a total of 18k images encompassing 44 defect types, VISION strives to mirror a wide range of real-world production scenarios. By supporting two ongoing challenge competitions on the VISION Datasets, we hope to foster further advancements in vision-based industrial inspection.

Figures (1)

• Figure 1: Overview of the VISION Datasets.