Bounding Box-Guided Diffusion for Synthesizing Industrial Images and Segmentation Map

TL;DR

This paper tackles the lack of labeled data for industrial defect segmentation by introducing a diffusion-based data synthesis pipeline conditioned on enriched bounding box representations to jointly generate RGB images and semantic maps. It models the joint distribution $p(I_n, S_n)$ conditioned on bounding boxes $B_n$, and introduces BASD and CBASD encodings along with an analog-bit segmentation encoding to enforce spatial and semantic fidelity. Key contributions include the BASD/CBASD conditioning, a simple denoising objective for the diffusion model, and two evaluation metrics SAE and EBR to quantify layout-consistency. Experiments on Wood Defect data show that synthetic data from the proposed method yields better segmentation performance than a layout-conditioned baseline, and combining real and synthetic data provides the best results. Overall, the work demonstrates that diffusion-based synthesis can bridge the gap between synthetic and real industrial data, enabling cost-efficient, robust segmentation models, with code publicly available.

Abstract

Synthetic dataset generation in Computer Vision, particularly for industrial applications, is still underexplored. Industrial defect segmentation, for instance, requires highly accurate labels, yet acquiring such data is costly and time-consuming. To address this challenge, we propose a novel diffusion-based pipeline for generating high-fidelity industrial datasets with minimal supervision. Our approach conditions the diffusion model on enriched bounding box representations to produce precise segmentation masks, ensuring realistic and accurately localized defect synthesis. Compared to existing layout-conditioned generative methods, our approach improves defect consistency and spatial accuracy. We introduce two quantitative metrics to evaluate the effectiveness of our method and assess its impact on a downstream segmentation task trained on real and synthetic data. Our results demonstrate that diffusion-based synthesis can bridge the gap between artificial and real-world industrial data, fostering more reliable and cost-efficient segmentation models. The code is publicly available at https://github.com/covisionlab/diffusion_labeling.

Figures (4)

• Figure 1: Overview of the proposed diffusion-based approach that generates both RGB and segmentation map in industrial setting.
• Figure 2: An overview of the proposed method: the user produces low-cost bounding box annotations which are then converted in two representations (BASD and C-BASD). Later, these encodings, are fed into the diffusion to condition the generation of both high quality RGB and segmentation masks of wood defects.
• Figure 3: Some samples of the Wood Defect Detection openwood with the semantic segmentation labels. The wood defects are the following: knot (blue), crack (red), quartzite (green), resin (yellow), marrow (magenta).
• Figure 4: Qualitative comparison between our method and Layout Diffusion layoutdiffusion. Each method shows the generated RGB image with respect to the bounding box condition - the same for both methods - and the overlapped defect segmentation map. The wood defects are the following: knot (blue), crack (red), quartzite (green), resin (yellow), marrow (magenta).