Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era

TL;DR

Industrial Anomaly Detection (IAD) benchmarks have been restricted in category diversity and scale, limiting evaluation of unified and cross-domain models. This paper introduces Real-IAD Variety, a large-scale benchmark with 160 categories across 28 industries, 24 materials, and 22 colors, totaling 198,960 high-resolution images with pixel-level annotations, plus rigorous MUAD, MVAD, and ZS/FSAD evaluation protocols. Through extensive experiments, MUAD methods degrade substantially as category count grows, while vision-language zero-/few-shot approaches show robust generalization with minimal sensitivity to category scale, demonstrating a path toward scalable, general-purpose industrial anomaly detectors. Real-IAD Variety thus provides a critical resource for training and evaluating next-generation foundation models in IAD, enabling cross-domain transfer, multi-view robustness, and multimodal reasoning for real-world industrial settings.

Abstract

Industrial Anomaly Detection (IAD) is critical for enhancing operational safety, ensuring product quality, and optimizing manufacturing efficiency across global industries. However, the IAD algorithms are severely constrained by the limitations of existing public benchmarks. Current datasets exhibit restricted category diversity and insufficient scale, frequently resulting in metric saturation and limited model transferability to real-world scenarios. To address this gap, we introduce Real-IAD Variety, the largest and most diverse IAD benchmark, comprising 198,960 high-resolution images across 160 distinct object categories. Its diversity is ensured through comprehensive coverage of 28 industries, 24 material types, and 22 color variations. Our comprehensive experimental analysis validates the benchmark's substantial challenge: state-of-the-art multi-class unsupervised anomaly detection methods experience significant performance degradation when scaled from 30 to 160 categories. Crucially, we demonstrate that vision-language models exhibit remarkable robustness to category scale-up, with minimal performance variation across different category counts, significantly enhancing generalization capabilities in diverse industrial contexts. The unprecedented scale and complexity of Real-IAD Variety position it as an essential resource for training and evaluating next-generation foundation models for anomaly detection. By providing this comprehensive benchmark with rigorous evaluation protocols across multi-class unsupervised, multi-view, and zero-/few-shot settings, we aim to accelerate research beyond domain-specific constraints, enabling the development of scalable, general-purpose anomaly detection systems. Real-IAD Variety will be made publicly available to facilitate innovation in this critical field.

Figures (4)

• Figure 1: Industry distribution of the proposed Real-IAD Variety dataset. The dataset encompasses 8 major industrial categories (denoted as c): electrical, transport, cultural products, metal, general, electronics, rubber plastic, and other manufacturing sectors. These major categories are further subdivided into 28 industrial subcategories (denoted as s). Complete nomenclature details are provided in Appendix.
• Figure 2: Data collection and annotation pipeline for the proposed Real-IAD Variety. The pipeline comprises a four-stage sequential process: (a) Material Preparation: This initial phase encompasses the assembly of a diverse array of materials, spanning 160 distinct categories sourced from 28 industrial domains and encompassing 24 material compositions. (b) Acquisition Equipment Design: The second phase involves the design of data capture apparatus, comprising one $\text{top}-\text{down}$ camera for overhead views and four peripheral cameras to capture lateral perspectives. (c) Data Collection and Annotation: The third phase pertains to the data collection process, which includes meticulous pixel-level manual annotation, rigorous algorithmic cross-validation, and iterative refinement. This process iterates until the model's predicted Average Precision (AP) scores exhibit negligible variation below a predetermined threshold, following the methodology established in Real-IAD realiad. (d) Defect Taxonomy: The lower section illustrates 23 distinct defect types alongside their characteristic visual representations. Zoom in for enhanced visibility of defect regions delineated in red.
• Figure 3: Statistical characteristics of Real-IAD Variety across multiple dimensions. (a) Anomalous region proportion: Real-IAD Variety exhibits a broader and more balanced distribution of anomalous region proportions relative to total image area compared to Real-IAD realiad, substantially increasing dataset complexity. (b) Defect aspect ratio: Real-IAD Variety provides diverse aspect ratios for minimum bounding rectangles of defects, comparable to Real-IAD, introducing additional diversity and detection challenges. Representative samples are shown for intuitive visualization. (c) Material distribution: Real-IAD Variety encompasses 24 material types for practical applications, imposing higher requirements on method robustness. (d) Color distribution: Real-IAD Variety captures a wide color spectrum, which is essential for color-based anomaly detection research.
• Figure 4: Performance trends of I-AUROC and P-AUPR metrics with increasing categories for MUAD, ZSAD and FSAD methods.