Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Texture-AD: An Anomaly Detection Dataset and Benchmark for Real Algorithm Development

Tianwu Lei, Bohan Wang, Silin Chen, Shurong Cao, Ningmu Zou

TL;DR

The experimental results show that Texture-AD is a difficult challenge for state-of-the-art algorithms and to adapt to diverse products in automated pipelines, a new evaluation method and results of baseline algorithms are presented.

Abstract

Anomaly detection is a crucial process in industrial manufacturing and has made significant advancements recently. However, there is a large variance between the data used in the development and the data collected by the production environment. Therefore, we present the Texture-AD benchmark based on representative texture-based anomaly detection to evaluate the effectiveness of unsupervised anomaly detection algorithms in real-world applications. This dataset includes images of 15 different cloth, 14 semiconductor wafers and 10 metal plates acquired under different optical schemes. In addition, it includes more than 10 different types of defects produced during real manufacturing processes, such as scratches, wrinkles, color variations and point defects, which are often more difficult to detect than existing datasets. All anomalous areas are provided with pixel-level annotations to facilitate comprehensive evaluation using anomaly detection models. Specifically, to adapt to diverse products in automated pipelines, we present a new evaluation method and results of baseline algorithms. The experimental results show that Texture-AD is a difficult challenge for state-of-the-art algorithms. To our knowledge, Texture-AD is the first dataset to be devoted to evaluating industrial defect detection algorithms in the real world. The dataset is available at https://XXX.

Texture-AD: An Anomaly Detection Dataset and Benchmark for Real Algorithm Development

TL;DR

The experimental results show that Texture-AD is a difficult challenge for state-of-the-art algorithms and to adapt to diverse products in automated pipelines, a new evaluation method and results of baseline algorithms are presented.

Abstract

Anomaly detection is a crucial process in industrial manufacturing and has made significant advancements recently. However, there is a large variance between the data used in the development and the data collected by the production environment. Therefore, we present the Texture-AD benchmark based on representative texture-based anomaly detection to evaluate the effectiveness of unsupervised anomaly detection algorithms in real-world applications. This dataset includes images of 15 different cloth, 14 semiconductor wafers and 10 metal plates acquired under different optical schemes. In addition, it includes more than 10 different types of defects produced during real manufacturing processes, such as scratches, wrinkles, color variations and point defects, which are often more difficult to detect than existing datasets. All anomalous areas are provided with pixel-level annotations to facilitate comprehensive evaluation using anomaly detection models. Specifically, to adapt to diverse products in automated pipelines, we present a new evaluation method and results of baseline algorithms. The experimental results show that Texture-AD is a difficult challenge for state-of-the-art algorithms. To our knowledge, Texture-AD is the first dataset to be devoted to evaluating industrial defect detection algorithms in the real world. The dataset is available at https://XXX.
Paper Structure (22 sections, 6 figures, 4 tables)

This paper contains 22 sections, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset
  4. Data Generation
  5. Anomaly Detection Methods
  6. Synthesis-based Anomaly Detection
  7. Reconstruction-based Anomaly Detection
  8. Feature-Embedding Based Methods
  9. Benchmark
  10. Baseline Methods
  11. SimpleNet
  12. PyramidFlow
  13. Mean-Shift
  14. DRAEM
  15. MSFlow
  16. ...and 7 more sections

Figures (6)

  • Figure 1: Difference between existing evaluation methods and actual situation
  • Figure 2: Data Statistics (a)The cloth dataset consists of a total of $6283$ images, with $4569$ images in the training set and $1714$ images in the test set. (b)The wafer dataset consists of a total of $14861$ images, with $10525$ images in the training set and $4336$ images in the test set. (c)The metal plate dataset consists of a total of $21976$ images, with $13879$ images in the training set and $8097$ images in the test set.
  • Figure 3: Statistics of the percentage of the image area occupied by the anomaly region
  • Figure 4: Image acquisition and defect annotation processes. The Texture-AD images were captured using a high-resolution industrial camera (MV-CS200-10 GC). The optical scheme was altered by adjusting the position of the light source and the brightness of two light sources. The cloth images include both artificial and natural defects, while the wafer and metal plate images consist solely of natural defects. The defect annotation work for the images was performed using Labelme.
  • Figure 5: The comparison of the average Image-AUROC obtained by various algorithms on Texture-AD and MVTec
  • ...and 1 more figures