Unsupervised Continual Anomaly Detection with Contrastively-learned Prompt

TL;DR

The paper tackles unsupervised anomaly detection under continual class expansion in industrial settings, where labeled anomalies are scarce. It introduces UCAD, a framework that combines a Continual Prompting Module (CPM) with Structure-based Contrastive Learning (SCL), and leverages the Segment Anything Model (SAM) to learn task-aware prompts and structure-guided representations without supervision. A test-time, task-agnostic pipeline automatically identifies the current task, retrieves corresponding prompts and knowledge, and performs pixel-level anomaly detection and segmentation using a single model. Experiments on MVTec AD and VisA show substantial improvements over prior methods and establish a new benchmark for unsupervised continual AD and segmentation, highlighting UCAD’s practical impact for robust, scalable industrial inspection.

Abstract

Unsupervised Anomaly Detection (UAD) with incremental training is crucial in industrial manufacturing, as unpredictable defects make obtaining sufficient labeled data infeasible. However, continual learning methods primarily rely on supervised annotations, while the application in UAD is limited due to the absence of supervision. Current UAD methods train separate models for different classes sequentially, leading to catastrophic forgetting and a heavy computational burden. To address this issue, we introduce a novel Unsupervised Continual Anomaly Detection framework called UCAD, which equips the UAD with continual learning capability through contrastively-learned prompts. In the proposed UCAD, we design a Continual Prompting Module (CPM) by utilizing a concise key-prompt-knowledge memory bank to guide task-invariant `anomaly' model predictions using task-specific `normal' knowledge. Moreover, Structure-based Contrastive Learning (SCL) is designed with the Segment Anything Model (SAM) to improve prompt learning and anomaly segmentation results. Specifically, by treating SAM's masks as structure, we draw features within the same mask closer and push others apart for general feature representations. We conduct comprehensive experiments and set the benchmark on unsupervised continual anomaly detection and segmentation, demonstrating that our method is significantly better than anomaly detection methods, even with rehearsal training. The code will be available at https://github.com/shirowalker/UCAD.

Figures (4)

• Figure 1: Comparison between separate models and UCAD methods: a) Using separate methods, each task has its own individual model. On the contrary, Ours b) uses a single model to handle all tasks without task identities. In the continuous stream, UCAD only requires the dataset of the current task for training and can be applied to previous tasks.
• Figure 2: The framework of UCAD mainly comprises a Continual Prompting Module (CPM) and a Structure-based Contrastive Learning (SCL) module, integrated with the SAM network. During training, the CPM establishes a key-prompt-knowledge system that efficiently maintains training data information, while also reducing memory and computational resource usage. Moreover, UCAD proposes a contrastive learning method using the SAM segmentation map to enhance the feature representations. Finally, the detection of anomalies is accomplished by comparing current features and retrieved task-specific knowledge.
• Figure 3: Visualization examples of continual anomaly detection. The first row displays the original anomaly images, the second row shows the ground truth annotations, and the third to fifth rows depict the heatmaps of our method and other methods.
• Figure 4: Visualization examples of continual anomaly detection. The first row displays the original anomaly images, the second row shows the ground truth annotations, and the third to fifth rows depict the heatmaps of our method and other methods.