Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Detect Multi-class Anomalies with Just One Normal Image Prompt

Bin-Bin Gao

TL;DR

The paper tackles unified multi-class anomaly detection with a single model by addressing reconstruction shortcomings that rely on context alone. It introduces OneNIP, a framework that uses a normal image as a visual prompt to guide feature reconstruction, supplemented by a restoration stream with pseudo anomalies and a supervised refiner for improved pixel-level segmentation, all within a bidirectional cross-attention architecture. The approach achieves state-of-the-art results on industrial benchmarks MVTec, BTAD, and VisA and demonstrates faster convergence than prior unified-model methods. This yields a practical, scalable solution for robust anomaly detection and segmentation in diverse industrial settings, with potential applicability beyond manufacturing.

Abstract

Unsupervised reconstruction networks using self-attention transformers have achieved state-of-the-art performance for multi-class (unified) anomaly detection with a single model. However, these self-attention reconstruction models primarily operate on target features, which may result in perfect reconstruction for both normal and anomaly features due to high consistency with context, leading to failure in detecting anomalies. Additionally, these models often produce inaccurate anomaly segmentation due to performing reconstruction in a low spatial resolution latent space. To enable reconstruction models enjoying high efficiency while enhancing their generalization for unified anomaly detection, we propose a simple yet effective method that reconstructs normal features and restores anomaly features with just One Normal Image Prompt (OneNIP). In contrast to previous work, OneNIP allows for the first time to reconstruct or restore anomalies with just one normal image prompt, effectively boosting unified anomaly detection performance. Furthermore, we propose a supervised refiner that regresses reconstruction errors by using both real normal and synthesized anomalous images, which significantly improves pixel-level anomaly segmentation. OneNIP outperforms previous methods on three industry anomaly detection benchmarks: MVTec, BTAD, and VisA. The code and pre-trained models are available at https://github.com/gaobb/OneNIP.

Learning to Detect Multi-class Anomalies with Just One Normal Image Prompt

TL;DR

The paper tackles unified multi-class anomaly detection with a single model by addressing reconstruction shortcomings that rely on context alone. It introduces OneNIP, a framework that uses a normal image as a visual prompt to guide feature reconstruction, supplemented by a restoration stream with pseudo anomalies and a supervised refiner for improved pixel-level segmentation, all within a bidirectional cross-attention architecture. The approach achieves state-of-the-art results on industrial benchmarks MVTec, BTAD, and VisA and demonstrates faster convergence than prior unified-model methods. This yields a practical, scalable solution for robust anomaly detection and segmentation in diverse industrial settings, with potential applicability beyond manufacturing.

Abstract

Unsupervised reconstruction networks using self-attention transformers have achieved state-of-the-art performance for multi-class (unified) anomaly detection with a single model. However, these self-attention reconstruction models primarily operate on target features, which may result in perfect reconstruction for both normal and anomaly features due to high consistency with context, leading to failure in detecting anomalies. Additionally, these models often produce inaccurate anomaly segmentation due to performing reconstruction in a low spatial resolution latent space. To enable reconstruction models enjoying high efficiency while enhancing their generalization for unified anomaly detection, we propose a simple yet effective method that reconstructs normal features and restores anomaly features with just One Normal Image Prompt (OneNIP). In contrast to previous work, OneNIP allows for the first time to reconstruct or restore anomalies with just one normal image prompt, effectively boosting unified anomaly detection performance. Furthermore, we propose a supervised refiner that regresses reconstruction errors by using both real normal and synthesized anomalous images, which significantly improves pixel-level anomaly segmentation. OneNIP outperforms previous methods on three industry anomaly detection benchmarks: MVTec, BTAD, and VisA. The code and pre-trained models are available at https://github.com/gaobb/OneNIP.
Paper Structure (16 sections, 10 equations, 3 figures, 6 tables)

This paper contains 16 sections, 10 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Reconstruction with Normal Image Prompt
  5. Restoration with Normal Image Prompt
  6. Supervised Refiner
  7. Training Loss
  8. Inference
  9. Experiment
  10. Experimental Setup
  11. Comparisons with State-of-the-Arts
  12. Ablation Studies
  13. Conclusion
  14. Implementation Details
  15. Industry Anomaly Detection Benchmarks
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Comparisons of state-of-the-art UinAD and our OneNIP. The proposed OneNIP detects anomalies through learning comparison with one normal image as a visual prompt. Compared to UniAD, OneNIP enjoys more accurate anomaly localization (a) and faster convergence (b).
  • Figure 2: Overview of OneNIP for unified anomaly detection. In the training stage, both normal and synthetic images are fed a pre-trained backbone for extracting multi-level representation. Under the guidance of a normal image prompt, the normal features are reconstructed in an unsupervised reconstruction stream (\ref{['sec:rec']}), and the synthetic anomaly features are restored in an unsupervised restoration stream (\ref{['sec:res']}). Furthermore, a supervised refiner (\ref{['sec:ref']}) is used to regress reconstruction errors for both normal and synthetic anomaly images. The unsupervised restoration stream will be removed at inference.
  • Figure 3: Qualitative comparisons of https://github.com/zhiyuanyou/UniAD (second and sixth rows) and our OneNIP (third and seventh rows) on MVTec (15 classes), BTAD (3 classes) and VisA (12 classes). Here, the first and fifth rows are original testing images, and the fourth and eighth rows are their corresponding anomaly masks highlighted with red color.