Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAD: A Comprehensive Dataset for Benchmarking the Robustness of Image Anomaly Detection

Yuqi Cheng, Yunkang Cao, Rui Chen, Weiming Shen

TL;DR

RAD introduces a dataset keystone for evaluating robustness in image anomaly detection under realistic imaging noise, including free views, uneven illumination, and blur. It benchmarks 11 unsupervised and zero-shot methods, revealing that performance degrades differently across noise types and that memory-bank-based approaches and synthetic anomalies offer stronger resilience, with foundation-model knowledge showing promise for further gains. The analysis identifies persistent challenges from background misalignment and lighting variations, and suggests that leveraging synthesized anomalies and foundation-model features can enhance practical robustness. The dataset and findings provide a practical benchmark to guide the development of robust anomaly detection for industrial inspection.

Abstract

Robustness against noisy imaging is crucial for practical image anomaly detection systems. This study introduces a Robust Anomaly Detection (RAD) dataset with free views, uneven illuminations, and blurry collections to systematically evaluate the robustness of current anomaly detection methods. Specifically, RAD aims to identify foreign objects on working platforms as anomalies. The collection process incorporates various sources of imaging noise, such as viewpoint changes, uneven illuminations, and blurry collections, to replicate real-world inspection scenarios. Subsequently, we assess and analyze 11 state-of-the-art unsupervised and zero-shot methods on RAD. Our findings indicate that: 1) Variations in viewpoint, illumination, and blurring affect anomaly detection methods to varying degrees; 2) Methods relying on memory banks and assisted by synthetic anomalies demonstrate stronger robustness; 3) Effectively leveraging the general knowledge of foundational models is a promising avenue for enhancing the robustness of anomaly detection methods. The dataset is available at https://github.com/hustCYQ/RAD-dataset.

RAD: A Comprehensive Dataset for Benchmarking the Robustness of Image Anomaly Detection

TL;DR

RAD introduces a dataset keystone for evaluating robustness in image anomaly detection under realistic imaging noise, including free views, uneven illumination, and blur. It benchmarks 11 unsupervised and zero-shot methods, revealing that performance degrades differently across noise types and that memory-bank-based approaches and synthetic anomalies offer stronger resilience, with foundation-model knowledge showing promise for further gains. The analysis identifies persistent challenges from background misalignment and lighting variations, and suggests that leveraging synthesized anomalies and foundation-model features can enhance practical robustness. The dataset and findings provide a practical benchmark to guide the development of robust anomaly detection for industrial inspection.

Abstract

Robustness against noisy imaging is crucial for practical image anomaly detection systems. This study introduces a Robust Anomaly Detection (RAD) dataset with free views, uneven illuminations, and blurry collections to systematically evaluate the robustness of current anomaly detection methods. Specifically, RAD aims to identify foreign objects on working platforms as anomalies. The collection process incorporates various sources of imaging noise, such as viewpoint changes, uneven illuminations, and blurry collections, to replicate real-world inspection scenarios. Subsequently, we assess and analyze 11 state-of-the-art unsupervised and zero-shot methods on RAD. Our findings indicate that: 1) Variations in viewpoint, illumination, and blurring affect anomaly detection methods to varying degrees; 2) Methods relying on memory banks and assisted by synthetic anomalies demonstrate stronger robustness; 3) Effectively leveraging the general knowledge of foundational models is a promising avenue for enhancing the robustness of anomaly detection methods. The dataset is available at https://github.com/hustCYQ/RAD-dataset.
Paper Structure (17 sections, 5 figures, 3 tables)

This paper contains 17 sections, 5 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. DATASET
  3. Data collecting
  4. Free View
  5. Uneven Illuminations
  6. Blurry Collections
  7. EXPERIMENT
  8. Experiment Settings
  9. Comparison Methods
  10. Evaluation Metrics
  11. Implementation Details
  12. Experiment Results
  13. Experiment Analysis
  14. Free Views
  15. Uneven Illuminations
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Samples from RAD dataset. The first row contains some normal samples. The second row contains some abnormal samples and the categories of foreign objects from left to right are bolt, cable tie, sponge, and tape.
  • Figure 2: Foreign objects used in data collection: bolt, cable tie, sponge, and tape.
  • Figure 3: Original images and corresponding pixel-wise masks from RAD dataset.
  • Figure 4: Samples from RAD dataset. (Left) Foreign object images collected from different perspectives. (Middle) Foreign object images with uneven illumination. (Right) Foreign object images with blurry.
  • Figure 5: Visualization of prediction results using the selected methods. Every three columns represent a category, with the first column representing high-quality sample, the second column representing samples with uneven illumination, and the third column representing samples with blurry. The first row is the original images collected, while the second row is the pixel-wise mask. Subsequent rows depict various methods: a) Cflow b) PyramidFlow c) RD d) RD++ e) DSR f) CDO g) Patchcore h) GCPF i) SAA j) WinCLIP k) APRIL-GAN.