Diffusion-based Image Generation for In-distribution Data Augmentation in Surface Defect Detection

TL;DR

The paper addresses the challenge of data scarcity for defective samples in industrial surface inspection by identifying flaws in standard augmentation that rely on out-of-distribution artifacts. It proposes In&Out, a hybrid augmentation framework that merges diffusion-based in-distribution defect generation via DDPMs (Dreambooth+LoRA) with per-region out-of-distribution augmentation, enabling robust learning of genuine defect appearances. Evaluated on the Kolektor Surface-Defect Dataset 2, In&Out achieves a new state-of-the-art AP of $0.782$ under weak supervision, with strong results across zero-shot and N-shot regimes and a practical code release. The approach demonstrates that combining realistic diffusion-generated defects with targeted patch-based augmentations can substantially improve recall and overall defect-detection performance, offering a scalable path for industrial deployment.

Abstract

In this study, we show that diffusion models can be used in industrial scenarios to improve the data augmentation procedure in the context of surface defect detection. In general, defect detection classifiers are trained on ground-truth data formed by normal samples (negative data) and samples with defects (positive data), where the latter are consistently fewer than normal samples. For these reasons, state-of-the-art data augmentation procedures add synthetic defect data by superimposing artifacts to normal samples. This leads to out-of-distribution augmented data so that the classification system learns what is not a normal sample but does not know what a defect really is. We show that diffusion models overcome this situation, providing more realistic in-distribution defects so that the model can learn the defect's genuine appearance. We propose a novel approach for data augmentation that mixes out-of-distribution with in-distribution samples, which we call In&Out. The approach can deal with two data augmentation setups: i) when no defects are available (zero-shot data augmentation) and ii) when defects are available, which can be in a small number (few-shot) or a large one (full-shot). We focus the experimental part on the most challenging benchmark in the state-of-the-art, i.e., the Kolektor Surface-Defect Dataset 2, defining the new state-of-the-art classification AP score under weak supervision of .782. The code is available at https://github.com/intelligolabs/in_and_out.

Figures (7)

• Figure 1: Idea underlying our In&Out data augmentation approach. (Left, blue dots) The blue dots outside the bulk of negative data could be wrongly classified as anomalies (false positives), being slightly different from most of the negative data. (Right, yellow crosses) State-of-the-art per-region data augmentation methods (for example, MemSeg yang2023memseg) add positive synthetic samples in that zone, which helps in deciding what is certainly not anomalous data. (Left, red dots) On the other hand, the red dot partially outside the bulk of positive data could be, in principle, understood as a negative sample, leading to a false negative. (Right, red crosses) Diffusion-based generated data is capable of producing defects very similar to the ones in the bulk of positive data, helping the classifier not produce false negative classifications.
• Figure 2: Augmented images generated by the MemSeg yang2023memseg pipeline. It is evident how it provides out-of-distribution positive samples.
• Figure 3: General schema of our In&Out method.
• Figure 4: Normal (top row) and anomalous (bottom row) samples from the KSDD2 dataset. Note that some defects are very difficult to find.
• Figure 5: Anomalous samples generated by DDPM. It is evident how it provides in-distribution positive samples.