Anomaly Detection by Effectively Leveraging Synthetic Images

TL;DR

Problem: Anomaly detection in manufacturing suffers from scarce real defect data. Approach: A training-free defect synthesis framework combines a pre-trained text-guided image-to-image translator with an image retrieval filter to produce high-fidelity defects, followed by a two-stage training that pre-trains on large-scale rule-based synthetic data and fine-tunes on high-quality synthetic samples. Contributions: a retrieval-based filtering mechanism to ensure structural consistency and a cost-efficient hybrid training strategy that improves AUROC on the MVTec AD dataset. Findings: the method achieves strong performance with reduced data-generation cost, demonstrating practical viability for deployment in data-scarce industrial settings.

Abstract

Anomaly detection plays a vital role in industrial manufacturing. Due to the scarcity of real defect images, unsupervised approaches that rely solely on normal images have been extensively studied. Recently, diffusion-based generative models brought attention to training data synthesis as an alternative solution. In this work, we focus on a strategy to effectively leverage synthetic images to maximize the anomaly detection performance. Previous synthesis strategies are broadly categorized into two groups, presenting a clear trade-off. Rule-based synthesis, such as injecting noise or pasting patches, is cost-effective but often fails to produce realistic defect images. On the other hand, generative model-based synthesis can create high-quality defect images but requires substantial cost. To address this problem, we propose a novel framework that leverages a pre-trained text-guided image-to-image translation model and image retrieval model to efficiently generate synthetic defect images. Specifically, the image retrieval model assesses the similarity of the generated images to real normal images and filters out irrelevant outputs, thereby enhancing the quality and relevance of the generated defect images. To effectively leverage synthetic images, we also introduce a two stage training strategy. In this strategy, the model is first pre-trained on a large volume of images from rule-based synthesis and then fine-tuned on a smaller set of high-quality images. This method significantly reduces the cost for data collection while improving the anomaly detection performance. Experiments on the MVTec AD dataset demonstrate the effectiveness of our approach.

Figures (23)

• Figure 1: Overview of the proposed approach. (a) To effectively leverage synthetic images, we introduce a novel training strategy that sequentially leverages large-scale rule-based synthesis for pre-training, followed by small-scale generative model-based synthesis for fine-tuning. (b) To provide high quality data for the fine-tuning stage, we also propose an effective anomaly synthesis framework. It consists of a pre-trained text-guided image-to-image translation model magicbrush and a pre-trained image retrieval model lightglue. Given a normal image and a prompt that describes defect, the pre-trained text-guided image-to-image translation model synthesizes a synthetic anomaly. Since the text-guided image-to-image translation model in this work is pre-trained on MS COCO dataset coco, it occasionally generates irrelevant results, we additionally employ a filtering mechanism. The image retrieval model compares the normal image with the synthetic anomaly, filtering out irrelevant synthetic anomalies based on the number of matched points.
• Figure 2: Examples of synthetic defect images that are translated from a normal image. The 'Desired anomaly cases' demonstrate high-fidelity synthesis where the generated defects are seamlessly integrated into the object surface while preserving the structural integrity and background context of the original normal image. In contrast, the 'Irrelevant anomaly cases' exhibit significant failure modes, such as severe background distortions, structural inconsistencies, and the introduction of irrelevant artifacts that deviate from realistic defect characteristics. While integrating high-quality synthetic samples into the training pipeline is crucial for enhancing the anomaly detection performance in data-scarce regimes, training on inconsistent data disrupts the learning of discriminative features, leading to model degradation. These observations underscore the critical necessity of a filtering mechanism to utilize high-fidelity synthesis.
• Figure 3: Visualization of the filtering mechanism based on feature matching. We compare the normal image with the synthetic anomaly. The underlying hypothesis is that a well-generated defect image should retain high structural correspondence with the original image, except for the localized defect region. The green lines represent the matching points between the two images. In the 'No anomaly cases', the image remains almost identical to the original, resulting in the highest number of matching points. In the 'Desired anomaly cases', fewer points are matched than in the 'No anomaly cases' due to the generated defects. However, there are still many matching points, meaning the structure information is well preserved. Finally, in the 'Irrelevant anomaly cases', there are very few matching points. This indicates that we can filter out meaningless synthetic defect images based on the number of matching points.
• Figure 4: Qualitative comparison of defect image synthesis methods across various object categories from the MVTec AD dataset mvtec. From left to right: DRAEM draem, Cut-Paste cutpaste, NSA nsa, RealNet realnet, AnomalyDiffusion anomalydiffusion, DFMGAN dfmgan, and ours. Defect images synthesized using rule-based synthesis are often unrealistic and do not resemble defects that can occur in real-world industrial manufacturing environments. In contrast, generative model-based synthesis approaches generally produce more realistic defect images. Our proposed method, despite relying solely on pre-trained models, is capable of generating defect images that are even more realistic than those produced by existing generative synthesis methods.
• Figure 5: Overview of the training pipeline. Blue boxes represent defect images generated by rule-based synthesis, while red boxes represent generative model-based synthetic images. Strategies (a), (b), and (c) use only one-stage training, whereas strategies (d) and (e) employ a two-stage training process with fine-tuning.