Few-Shot Anomaly-Driven Generation for Anomaly Classification and Segmentation

TL;DR

Industrial anomaly detection suffers from scarcity of real anomaly samples and a semantic gap when using synthetic anomalies. The authors propose AnoGen, a three-stage framework that learns a compact anomaly embedding from a few support anomalies, uses bounding-box conditioned diffusion to generate realistic, diverse anomalies, and trains a weakly supervised detector with bounding-box supervision. On the MVTec dataset, AnoGen improves both anomaly classification and segmentation for baseline models (e.g., DRAEM and DeSTSeg), achieving notable gains in pixel-level AU-PR. The work demonstrates that realism and spatial controllability in generated anomalies, coupled with weak supervision, substantially enhances practical anomaly detection with limited anomalous data.

Abstract

Anomaly detection is a practical and challenging task due to the scarcity of anomaly samples in industrial inspection. Some existing anomaly detection methods address this issue by synthesizing anomalies with noise or external data. However, there is always a large semantic gap between synthetic and real-world anomalies, resulting in weak performance in anomaly detection. To solve the problem, we propose a few-shot Anomaly-driven Generation (AnoGen) method, which guides the diffusion model to generate realistic and diverse anomalies with only a few real anomalies, thereby benefiting training anomaly detection models. Specifically, our work is divided into three stages. In the first stage, we learn the anomaly distribution based on a few given real anomalies and inject the learned knowledge into an embedding. In the second stage, we use the embedding and given bounding boxes to guide the diffusion model to generate realistic and diverse anomalies on specific objects (or textures). In the final stage, we propose a weakly-supervised anomaly detection method to train a more powerful model with generated anomalies. Our method builds upon DRAEM and DesTSeg as the foundation model and conducts experiments on the commonly used industrial anomaly detection dataset, MVTec. The experiments demonstrate that our generated anomalies effectively improve the model performance of both anomaly classification and segmentation tasks simultaneously, \eg, DRAEM and DseTSeg achieved a 5.8\% and 1.5\% improvement in AU-PR metric on segmentation task, respectively. The code and generated anomalous data are available at https://github.com/gaobb/AnoGen.

Figures (9)

• Figure 1: Comparisons of real anomalies (left column) and generated anomalies with ours (middle column) and other methods (right column). Given a few images of a real anomaly concept, our AnoGen is able to generate more realistic and diverse anomalies through learning a pre-trained diffusion model compared to the existing synthetic methods such as DRAEM and CutPaste. Meanwhile, our generated anomalies are spatially controllable because of a given mask (e.g., bounding box), which will benefit downstream anomaly detection tasks, i.e., classification and segmentation.
• Figure 2: Pipeline of our work, and it consists of three stages. In the first stage, we learn an embedding vector $\boldsymbol{v}$ with few support anomalies ($I_a^T$, $M_a^T$) based on a pre-trained Latent Diffusion Model (LDM) fixing all parameters, where the number of real-world anomalous images $I_a^T$ is only 1 or 3, and $M_a^T$ is the corresponding ground-truth masks. In the second stage, given a normal image $I_n$ and a bounding box mask $M^{box}$, we use the learned embedding $\boldsymbol{v}^*$ to guide the LDM to generate anomalous image $I_a^G$. In the third stage, we use the normal image $I_n$, bounding box mask $M^{box}$, and generated image $I_a^G$ to train a weakly-supervised anomaly detection model for image-level classification and pixel-level segmentation.
• Figure 3: We show six sets of images, in each set, the first column is the support anomalies (only 3 images), and the second column is the object (or texture) sampled from the training set with a randomly generated bounding box mask, the third and fourth columns are the generated anomalous images.
• Figure 3: Ablation study of $\tau$.
• Figure 5: Ablation study of $N$.