ProtoAnomalyNCD: Prototype Learning for Multi-class Novel Anomaly Discovery in Industrial Scenarios

TL;DR

ProtoAnomalyNCD tackles open-set industrial anomaly classification by combining Grounded SAM localization, anomaly-map-guided attention, and a unified prototype-based learning framework on a hyperspherical feature space with von Mises–Fisher distributions. It jointly discovers unseen anomaly types, classifies multiple anomaly categories, estimates the number of novel classes with a prototype-scoring strategy, and extends to out-of-distribution rejection. Key contributions include the anomaly-prior integrated AMG-ViT, region-guided prototypes, corrected pseudo-labels, and a Kadapter-style open-set extension validated on MVTec AD, MTD, and Real-IAD with strong gains over state-of-the-art clustering and NCD baselines. The approach offers a practical, scalable solution for open-set, multi-type industrial anomaly analysis with potential impact on real-world quality assurance and predictive maintenance.

Abstract

Existing industrial anomaly detection methods mainly determine whether an anomaly is present. However, real-world applications also require discovering and classifying multiple anomaly types. Since industrial anomalies are semantically subtle and current methods do not sufficiently exploit image priors, direct clustering approaches often perform poorly. To address these challenges, we propose ProtoAnomalyNCD, a prototype-learning-based framework for discovering unseen anomaly classes of multiple types that can be integrated with various anomaly detection methods. First, to suppress background clutter, we leverage Grounded SAM with text prompts to localize object regions as priors for the anomaly classification network. Next, because anomalies usually appear as subtle and fine-grained patterns on the product, we introduce an Anomaly-Map-Guided Attention block. Within this block, we design a Region Guidance Factor that helps the attention module distinguish among background, object regions, and anomalous regions. By using both localized product regions and anomaly maps as priors, the module enhances anomalous features while suppressing background noise and preserving normal features for contrastive learning. Finally, under a unified prototype-learning framework, ProtoAnomalyNCD discovers and clusters unseen anomaly classes while simultaneously enabling multi-type anomaly classification. We further extend our method to detect unseen outliers, achieving task-level unification. Our method outperforms state-of-the-art approaches on the MVTec AD, MTD, and Real-IAD datasets.

Figures (5)

• Figure 1: Comparison between solutions organizing anomalies into groups. (a) Clustering-based methods extract features from anomaly regions and apply unsupervised clustering. (b) Vanilla NCD methods use a trainable feature extractor and classifier on object-centered images from both known and unknown classes. (c) Our ProtoAnomalyNCD learns anomaly prototypes directly from anomaly-centered crops and masks to perform classification.
• Figure 2: (a) Overview of the ProtoAnomalyNCD training pipeline. (b) Structure of the anomaly-map-guided attention module. (c) During inference, ProtoAnomalyNCD can classify both previously known and novel classes, and can also be extended to reject unseen outliers.
• Figure 3: Out-of-Distribution Detection Workflow for Industrial Anomaly Inspection
• Figure 4: Visualization of the self-attention of the [CLS] token on the last layer’s heads. DINO attention refers to the [CLS] token extracted from a DINO pre-trained ViT that mainly focuses on a foreground object. ProtoAnomalyNCD uses a anomaly map to direct the [CLS] token’s attention to the anomalous regions.
• Figure 5: T-SNE visualization of sub-images on the MVTec AD dataset. We choose leather, hazelnut and wood as examples. The different colors of dots represent their anomaly classes.