Leveraging Latent Diffusion Models for Training-Free In-Distribution Data Augmentation for Surface Defect Detection

TL;DR

This paper tackles defect detection under data scarcity by proposing DIAG, a training-free diffusion-based pipeline that generates in-distribution defect images guided by domain experts through text prompts and region localization. It leverages a Latent Diffusion Model with inpainting to produce plausible anomalies on defect-free samples, forming a diverse augmented set without fine-tuning. Framed as binary anomaly detection with a ResNet-50 backbone trained on negative images plus synthetic positives, DIAG demonstrates superior AP on the KSDD2 dataset in both zero-shot and full-shot scenarios, outperforming prior augmentation methods. Importantly, DIAG yields highly realistic defects, as reflected by favorable Fréchet Inception Distance (FID) scores, suggesting strong alignment with real-world defect distributions and practical potential for rapid adaptation in manufacturing settings.

Abstract

Defect detection is the task of identifying defects in production samples. Usually, defect detection classifiers are trained on ground-truth data formed by normal samples (negative data) and samples with defects (positive data), where the latter are consistently fewer than normal samples. State-of-the-art data augmentation procedures add synthetic defect data by superimposing artifacts to normal samples to mitigate problems related to unbalanced training data. These techniques often produce out-of-distribution images, resulting in systems that learn what is not a normal sample but cannot accurately identify what a defect looks like. In this work, we introduce DIAG, a training-free Diffusion-based In-distribution Anomaly Generation pipeline for data augmentation. Unlike conventional image generation techniques, we implement a human-in-the-loop pipeline, where domain experts provide multimodal guidance to the model through text descriptions and region localization of the possible anomalies. This strategic shift enhances the interpretability of results and fosters a more robust human feedback loop, facilitating iterative improvements of the generated outputs. Remarkably, our approach operates in a zero-shot manner, avoiding time-consuming fine-tuning procedures while achieving superior performance. We demonstrate the efficacy and versatility of DIAG with respect to state-of-the-art data augmentation approaches on the challenging KSDD2 dataset, with an improvement in AP of approximately 18% when positive samples are available and 28% when they are missing. The source code is available at https://github.com/intelligolabs/DIAG.

Figures (2)

• Figure 1: The DIAG pipeline. Starting from positive samples, we leverage a Latent Diffusion Model (LDM) to synthesize novel in-distribution high-quality images of defective surfaces based on defect localization and textual prompts. These synthetic images are then used as anomaly samples to train a binary classifier for anomaly detection.
• Figure 2: First row displays some negative samples from the KSDD2 dataset. Instead, the second row shows some images of positive samples from the same dataset. In the third row, we show the MemSeg-generated defect samples. The fourth row shows In&Out generated defect samples. Lastly, the final row showcases images generated with DIAG. Notably, the defect images that DIAG generated are more realistic and in-distribution.