Normal-Abnormal Guided Generalist Anomaly Detection

TL;DR

This work tackles cross-domain anomaly detection by leveraging a mixture of normal and abnormal references. It introduces the Normal-Abnormal Generalist Learning (NAGL) framework, composed of Residual Mining (RM) and Anomaly Feature Learning (AFL), to mine abnormal patterns from residuals and apply them to query images. Trained on the original domain with episodic normal-abnormal references, NAGL achieves state-of-the-art cross-domain detection and segmentation while maintaining efficiency, as shown on industrial (MVTecAD, VisA) and medical (BraTS) benchmarks. The results demonstrate robust generalization, effective use of scarce abnormal samples, and practical viability for real-world deployment in industrial inspection and medical diagnosis.

Abstract

Generalist Anomaly Detection (GAD) aims to train a unified model on an original domain that can detect anomalies in new target domains. Previous GAD methods primarily use only normal samples as references, overlooking the valuable information contained in anomalous samples that are often available in real-world scenarios. To address this limitation, we propose a more practical approach: normal-abnormal-guided generalist anomaly detection, which leverages both normal and anomalous samples as references to guide anomaly detection across diverse domains. We introduce the Normal-Abnormal Generalist Learning (NAGL) framework, consisting of two key components: Residual Mining (RM) and Anomaly Feature Learning (AFL). RM extracts abnormal patterns from normal-abnormal reference residuals to establish transferable anomaly representations, while AFL adaptively learns anomaly features in query images through residual mapping to identify instance-aware anomalies. Our approach effectively utilizes both normal and anomalous references for more accurate and efficient cross-domain anomaly detection. Extensive experiments across multiple benchmarks demonstrate that our method significantly outperforms existing GAD approaches. This work represents the first to adopt a mixture of normal and abnormal samples as references in generalist anomaly detection. The code and datasets are available at https://github.com/JasonKyng/NAGL.

Figures (8)

• Figure 1: Overview of existing and proposed GAD paradigms. (a) Vanilla GAD only adopts a few normal samples as references. (b) Our approach combines normal and abnormal references to enhance detection. (c) Direct application of the KNN-based method to our normal-abnormal guided detection task causes to false activations (middle), while our approach eliminates them (right). (d) Comparisons of different methods in terms of AUROC sample (y-axis), inference speed (x-axis), and model size (circle radius). Our approach achieves the highest AUROC sample metric for anomaly detection while being $\textbf{2}\times$faster than ResAD and $\textbf{14}\times$faster than InCTRL.
• Figure 2: Overview of our proposed NAGL framework. Given a test image and its corresponding reference images (normal and abnormal), features are extracted through a pre-trained backbone network. The extracted normal features guide the generation of a normal-guided score map. Meanwhile, abnormal features are processed through the RM-AFL module to produce an abnormal-guided score map. This module implements a transformation process: from learnable proxies in vision space to residual proxies capturing normal-abnormal differences in residual space, and finally to anomaly proxies in vision space that highlight specific anomalous regions in the query image. The final anomaly score is computed by merging both normal and abnormal guided score maps.
• Figure 3: Qualitative results. The first row displays the input images, and the second row shows the ground truth. The third row illustrates anomaly score maps using $1$ normal sample, while the bottom row shows anomaly score maps produced by our method using $1$ normal and $1$ abnormal samples.
• Figure 4: Some examples of our input data
• Figure 5: T-SNE visualization of features. Different colours denote different defect types. (a) In the original vision space, the features from different defect types are significantly different. (b) In the residual feature space, these features are more overlapped in their distribution.