ReplayCAD: Generative Diffusion Replay for Continual Anomaly Detection

TL;DR

ReplayCAD tackles continual anomaly detection by integrating diffusion-based generative replay with semantic and spatial conditioning to preserve pixel-level features of historical data. It compresses historical class data into semantic embeddings and spatial masks, then replays this controlled content to train anomaly detectors alongside new classes. The approach yields state-of-the-art segmentation and classification performance, with substantial gains on VisA and MVTec and greatly reduced storage requirements. By enabling high-quality, diverse replay data, ReplayCAD mitigates catastrophic forgetting and enhances pixel-level anomaly localization for industrial inspection. The method is readily adaptable to different anomaly detectors and offers scalable, diffusion-driven CAD suitable for real-world deployment.

Abstract

Continual Anomaly Detection (CAD) enables anomaly detection models in learning new classes while preserving knowledge of historical classes. CAD faces two key challenges: catastrophic forgetting and segmentation of small anomalous regions. Existing CAD methods store image distributions or patch features to mitigate catastrophic forgetting, but they fail to preserve pixel-level detailed features for accurate segmentation. To overcome this limitation, we propose ReplayCAD, a novel diffusion-driven generative replay framework that replay high-quality historical data, thus effectively preserving pixel-level detailed features. Specifically, we compress historical data by searching for a class semantic embedding in the conditional space of the pre-trained diffusion model, which can guide the model to replay data with fine-grained pixel details, thus improving the segmentation performance. However, relying solely on semantic features results in limited spatial diversity. Hence, we further use spatial features to guide data compression, achieving precise control of sample space, thereby generating more diverse data. Our method achieves state-of-the-art performance in both classification and segmentation, with notable improvements in segmentation: 11.5% on VisA and 8.1% on MVTec. Our source code is available at https://github.com/HULEI7/ReplayCAD.

Figures (14)

• Figure 1: Compared to the SOTA method UCAD, our approach achieves more accurate anomaly segmentation, since we better preserves pixel-level detailed features through high-quality data replay.
• Figure 2: Visualization of semantic and spatial features. (a) Semantic features, samples from different classes exhibit distinct texture and shape characteristics. (b) Spatial features, samples within the same class are distributed across different spatial locations.
• Figure 3: Visualization of anomalous regions, illustrating their typically small areas.
• Figure 4: Overview of Data Compression. We compress the data from both semantic and spatial dimensions: (1) Semantic. In the semantic encoder, we get the semantic feature by initializing a class semantic embedding $v$ and combining it with a prompt $p$ encoded by the text encoder. (2) Spatial. In the spatial encoder, we first obtain the mask of the sample using SAM and learn the mapping of mask to spatial features by optimizing a MLP layer. Finally, we use spatial and semantic features as conditions to guide the denoising process of LDM.
• Figure 5: Overview of Data Replay. We use stored semantic embeddings $v$, the mlp layer, and mask images to guide the LDM in replaying data from historical classes. To enhance the diversity of replay samples, we apply random augmentations to the masks.